r 


AD-A020  947 


DESIGN  AND  ANALYSIS  OF  FIXED  TIME  RELIABILITY 
DEMONSTRATION  TESTS 

J.  Adelsberg 

Naval  Air  Development  Center 
Warminster,  Pennsylvania 

1 November  1975 


DISTRIBUTED  BY: 

National  Teshnicai  information  Service 
U,  S.  DEPARTMENT  OF  COMMERCE 


J 


fDA0209  47 


058165 


REPORT  NO.  NADC-75265-60 


DESIGN  AND  ANALYSIS  OF 
FIXED  HME  REUASmr/  DEMONSTRATION  TESTS 


J.  Adelcberg 

Haval  Navigation  Laboratory 
mM  AIR  DEVELOPMENT  CENTER 
Viarminster,  Pennsylvania  18974 


1 November  1975 


PHASE  REPORT 
TASK  NO.  SRI 405002 
Work  Unit  No.  PT501 


APPROVED  FOR  PUBLIC  RELEASE,-  DISTRIBUTION  UNLIMITED. 


Prepared  for 

NA\  ...  SEA  SYSTEMS  COMMAND 
Department  of  the  Navy 
Washington,  D.  C.  20361 


R^p'odvced  by 


NATIONAL  TECHNICAL 
INFORMATION  SERVICE 

U S D«pcrfmen»  of  C©mmer<# 
Springfield  VA  22)51 


NOTICES 


REPORT  NUMBERING  SYSTEM  - The  numbtring  af  technicci  prcjKt  rtpom  issued  tsy  tht  Neva!  Air 
Davatof^mm  Cantar  is  arranged  for  specific  identification  purposes.  Each  number  consists  of  the  Canter 
acronym,  the  caiandar  year  in  whidi  the  number  was  asigi»d,  the  sequence  nunr±«r  of  the  report  within 
the  speeifie  caiandar  year,  and  the  offidal  2-digit  correspondence  code  of  the  Command  Office  or  the 
Functkxui  Oepa^ment  ra^nsibie  for  the  report.  For  example:  Report  No.  NAOC-7301540  indicates 
the  fifteenth  Center  report  for  the  year  1973.  er>d  prepared  by  the  Crew  Systems  Department.  The 
nunterieai  codes  are  as  follows: 

CODE  OFFICE  OR  DEPARTMENT 

00  Comrrwxisr  Naval  Air  Development  Center 

01  Technical  Director.  Neval  Air  Devaiepment  Center 

02  Program  ami  Financial  Man^emciit  Department 

03  Anti-Submvine  Warfare  Program  Office 

04  Remote  Sensors  Program  Office 

06  Ship  and  Air  Systems  Integration  Pro(^sm  Office 

06  Tactical  Air  Warfare  Offks 

10  Naval  Air  Facility.  Warminster 

20  Aara  Electronic  Technology  Department 

3C  Air  Vehicle  Technology  Department 

40  Crew  Systems  Department 

EO  Systems  Analysis  and  Engineertna  Department 

60  Naval  Navigation  laboratory 

81  .Administrative  and  Technical  Saratsxs  Dcperimeni 

85  Computer  Oepartnr»nt 


r 


SZCUKITY  CLASStriCATION  OF  THIS  FAOC  (Whti  DMm  EaffOj 


REPORT  DOCUMENTATION  PAGE 


. AEFOMT  HUMBEH 

NADC-75265-60 


4.  title  fWSulHIK*} 


Dejsign  and  Analysis  of  Fixed  Time  Reliability 
Demonstration  Tests 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


3.  RECIPIENT'S  CATALOG  NUMBER 


5.  TYPE  OF  REPORT  « PERIOD  COVERED 


Phase  Report 


8.  PERFORMING  ORC.  REPORT  HUMBER 


7.  AuTMOiir*; 


I.  contract  or  grant  NUM8CRC«; 


J.  Adelsberg 


t.  PERFORMING  organization  NAME  AND  ADDRESS 

Naval  Navigation  Department  (Code  60) 
Naval  Air  Development  Center 
Warminster,  Pa.  18974 


tl  controlling  CFrice  NAME  AND  ADDRESS 

Naval  Sea  Systems  Command 
Department  of  the  Navy 
Washington,  D.  C.  20360 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  8 WORK  UNIT  NUMBERS 

61153N/R1405/SR  1405002 
Work  Unit  PT501 


12.  REPORT  DATE 

1 November  1975 


13.  NUMRER  OF  PACES 

0\ 


MONITORING  AGENCY  NAME  * AOOPLSS<</ d<//«r«n(  /raoi  ConitolUnt  Otilcm)  IS.  SECURITY  CLASS,  (cl  ttilt  tcpcH) 

I 

Unclassified 


IS*.  DECLASSIFICATION/ DOWNGRAOINC 
SCHEDULE 


It.  OlSTRiauTiON  STATCMEKT  Cot  thio  Repcro 


Approved  for  Public  Release;  Distribution  Unlimited. 


17.  OlSTRituTlON  STATCMCNT  (of  /fi  Block  70,  it  from  Boport) 


If.  KEY  VOROt  (Cor^drwm  on  rovcfoo  oldo  tl  nosoooory  idonllfy  by  block  nimbor) 

Reliability 

Statistical  Demonstration  Tests 


20  abstract  (Conttnuo  on  •/<!•  If  nmcoooofr  Id^llfy  by  block  mmtbot) 


The  principal  reliability  demonstration  test  methods  are  identified  and  their 
characterist.es  reviewed.  The  use  of  a graphic  procedure  for  the  design  and 
analysis  of  fixed  time  tests  is  discussed  and  examples  illustrating  applica- 
tion to  the  design  of  Classical,  Bayes  and  Hybrid  plans  are  given.  The 
graphic  procedure  is  also  applied  to  evaluate  the  effect  on  the  Bayes  posteric 
risks  as  a function  of  different  prior  distributional  characteristics  and  to 
make  comparisons  between  the  different,  test  methods.  A technique  tor 


00  1 jAN*n  1^3  EDITION  OF  1 NOV  88  IT  0630LETE  * 
S/N  0I03-OJ4-68C.  ! f 


UNCLASSIFIED 

fCCURlTY  CLA81IFICATION  OF  THIS  PAGE  jURm  Due  Kittcnd) 


NADC-75265-60 


SUMMARY 


INT\CDUCTION 

Applicecion  of  the  classical  dezDonstratlon  method  tends  to  lead  to 
long  test**  when  the  mean  time  between  failures  (MTBF)  of  the  devlc"  to 
be  tested  is  large  and  few  samples  are  available.  The  B^’yes  technioue 
while  often  successful  in  shortening  tesr  time  requirements,  introduces 
other  difficulties.  The  most  significant  is  that  test  properties 
generally  are  quite  sensitive  to  the  characteristics  of  the  prior 
distribution.  When  the  prior  assignment  is  representative  of  the  MTBF 
characteristic  of  the  device  to  be  cest-sd  there  is  considerable  gain,  in 
terms  of  test  costs,  resulting  trom  the  use  of  the  Bayes  approach. 
However,  in  most  Instances  the  needed  data  and  data  accuracy  is  lacking, 
leading  to  uncertain  results. 

Attempts  to  mitigate  this  problem  have  led  to  the  development  of 
yet  another  technique,  denoted  as  the  Hybrid  Method.  Here  prior  infor- 
mation as  well  as  a classical  criterion  is  utilized  in  test  design, 
usually  in  the  form  of  a Bayes  producer's  risk  and  a classical  consumer  s 
i-isk.  This  combination  results  in  a number  of  attractive  t st  features. 
An  important  one,  especially  from  the  users  point  of  view,  ’s  that  the 
consumer  protection  is  Independent  of  the  prior  distribution  and  remains 
within  a specified  level.  The  producer  also  realizes  some  benefit  in 
that  he  is  able  to  incorporate  pertinent  information,  acquired  during 
development,  into  the  test  plan  design  and  thus  affect  test  properties 
such  as  producer's  risk  and  test  time.  Another  attribute  that  is  of 
mutual  benefit  is  that  a Hybrid  test  can  generally  be  performed  in  a 
shorter  time  period  than  a comparable  classical  plan.  The  exact  savings 
in  test  time  depend  on  the  characteristics  of  the  prior  distribution  and 
the  specific  test  parameter  values  selected,  but  can  be  readily  deter- 
mine.. with  the  aid  of  the  charts  provided  in  this  report. 

A primary  aim  of  this  study  was  to  investigate  some  consequences  of 
the  Bayes  method,  especially  those-  aspects  of  the  procedure  that  impact 
on  the  user,  i-e.,  the  Government.  Concern  had  been  voiced  about  the 
consumer  protection  provided  by  these  plans  and  there  was  apprehension 
that  this  approach  seemingly  led  to  the  acceptance  of  bad  material,  much 
in  excess  of  allowable  ei'rors.  Recent  applications  of  Bayes  tests  in 
which  test  time  was  extremely  short  or  completely  eliminated  tended  to 
reinforce  this  feeling.  The  crux-  of  the  problem  was  the  lack  of  a 
quantitative  technique  to  assess  the  impact  of  the  prior  information  on 
consumer  related  test  properties.  Thus,  an  important  element  of  this 
study  concerned  the  development  of  a method  capable  of  evaluating  these 
dependencies  for  different  choices  of  prior  information. 

A further  objective  was  to  develop  procedures  to  enable  the  design 
and  analysis  of  fixed  time  test  plans,  for  all  currently  used  methods, 
to  be  accomplished  in  an  efficient  manner.  This  objective  was  set  forth 
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because  of  the  practical  difficulties  that  presently  exist  in  the  imple- 
mentation of  a test  requirement,  especially  when  attempting  to  use  the 
newer  techniques.  The  pertinent  literature  often  lacks  the  needed  depth 
or  detail  to  permit  direct  application  to  test  design.  The  Intention, 
therefore,  was  to  provide  a compendium  of  user  oriented  material  suitable 
for  this  purpose. 

In  opite  of  these  seemingly  diverse  objectives,  solutions  were 
derived  using  a comiMn  approach-  The  key  element  in  this  approach  is  a 
graphics..  proc-»dure  that  proved  to  be  as  versatile  when  applied  for 
purposes  of  test  plan  design  as  it  was  in  the  analysis  of  these  plans. 

For  example,  with  the  aid  of  these  graphs,  it  became  a relatively  simple 
matter  to  compute  the  change  in  Bayes  consumer’s  and  producer's  risk 
resulting  from  adjustments  in  the  prior  parameters  while  keeping  other 
variables  fixed.  The  graphical  procedure  also  performed  well  when 
utilized  as  a design  tool.  It  provided  a simple,  accurate,  yet  flexible 
technique  in  the  construction  of  any  fixed  time  test  plan,  regardless  of 
method. 

Finally,  the  availability  of  alternate  test  methods,  while  intro- 
ducing additional  design  flexibility,  may  actually  complicate  the  task 
of  test  plan  selection  unless  some  logical  anti  systematic  scheme  is 
employed  for  this  function.  To  help  guide  this  effort,  the  report 
provides  a set  of  criteria  and  explains  how  they  can  be  applied  to 
a derive  a preferred  test  plan. 

CONCLUSIONS 

1.  A Bayes  test  is  a relatively  poor  method  for  detecting  inadequate 
devices  especially  when  the  prior  is  optimistic. 

b.  Use  of  an  optimistic  prior  in  a Bayes  test  will  generally  result  in 
a shoit  test,  but  it  will  also  give  rise  to  a dramatic  reduction  in  user 
protection  (i.e.,  the  protection  against  acceptance  of  inadequate 
devices. ) 

c.  Since  the  formal  test  period  in  a Bayes  plan  with  an  optimistic 
prior  is  Invariably  much  shorter  than  the  prior  .MTBF  estimate,  it 
represents  a negligible  factor  in  the  test  decision  and  its  use  should 
therefore  be  discontinued  (i.e.,  the  major  benefit  of  a test  of  this 
type  is  of  a psychological  rather  than  a statistical  nature). 

d.  The  suggested  use  of  the  Probability  of  Acceptance,  P (A  ) as  the 
producer's  criterion  in  a Bayes  test  in  lieu  of  the  Bayes  posterior 
producer's  risk  does  have  merit  in  that  it  represents  a more  meaningful 
crltarlon  for  a contractor.  However,  construction  of  Bayes  tests  using 
this  criterion  tends  to  lead  to  very  short  tests  with  attendant  loss  of 
consumer  protection  and  thus  should  be  avoided.  P(A  ) can,  however,  be 
constructively  used  in  another  manner;  it  can  be  employed  as  a technique 
for  predicting  the  reliability  status  of  a product  in  the  various  stages  of 
Its  development  and  thus  help  to  flag  problems  that  may  require  corrective 
action. 


il 


NADC-7S265-60 


w 

I 

t 

f- 


e.  More  wide  spread  application  of  the  Jayes  approach  dcpe:>ds  mainly 
on  the  development  of  more  realistic  priors.  One  way  this  can  be 
achieved  is  through  implementation  of  a reliability  growth  estimation 
procedure.  The  test  data  generated  during  systems  development  and 
utilized,  in  part,  to  satisfy  the  requirements  of  tracking  reliability 
growth  as  the  design  progresses,  could  also  serve  as  the  basis  for 
constructing  more  representative  priors.  (Instituting  requirements  for 
reliability  growth  modeling  would  not  only  achieve  much  better  control 
of  the  reliability  of  an  evolving  system,  but  would  also  provide  a 
representative  data  base  for  Bayes  type  demonstration  tests.) 

f.  At  present,  the  Hybrid  test  method  represents  the  most  cost  effective 
solution  to  a broad  spectrum  of  fixed  time  demonstration  test  problems 
and  should  therefore  be  specified  and  applied  more  frequently. 

RECOMMENDATIONS 


a.  The  portion  of  MIL  STD  781-B  dealing  with  fixed  time  tests  should  be 
updated  to  Incorporate  use  of  the  graphical  design  and  analysis  pro- 
cedure for  the  construction  of  classical,  Bayes  and  ilybrid  demonstration 
test  plans. 

b.  Prior  to  the  selection  and  specification  of  a test  plan,  the 
operating  characteristics  (O.C.)  data  of  the  proposed  plan  should  be 
generated  and  examined  to  ensure  chat  the  test  plan's  performance  is 
consistent  with  desired  test  objectives  (the  technique  described  in 
the  report  may  be  used  to  obtain  the  O.C.  data). 

c.  A program  aimed  at  familiarizing  user  groups  with  the  character- 
istics and  application  of  the  Hybrid  test  method  should  be  instituted. 

d.  The  following  additional  work  effort,  limited  to  tasks  which  are 
intended  to  bioaden  the  range  of  application  of  already  developed 
techniques,  if.  recommended. 

1)  Extend  the  Bayes  approach  to  accommodate  applications  where 
reliability  instead  of  MTBF  is  the  appropriate  success  criterion. 

Generally  this  applies  to  systems  that  operate  over  relatively  short 
time  periods  such  as  sonobuoys  or  missiles.  This  effort  would  produce 
results  similar  to  what  is  available  for  the  MTBF  case,  but  start  with  a 
more  suitable  prior  distribution. 

ii)  Evaluate  the  sensitivity  of  the  Bayes  posterior  risks  as  a 
function  of  the  parar»eter  values  of  the  prior  distribution.  Results  of 
this  study  will  provide  guidelines  on  how  coverage  of  a range  of  para- 
meter values  can  be  achieved  with  a suitably  small  number  of  graphs- 

ill)  Develop  a technique  for  constructing  the  Operating  Characteristics 
(O.C.)  curve  for  Bayes  sequential  tests.  A procedure  for  determining  the 

1 
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SECTION  I 

PREFACE 

Reliability  demonstration  testing  is  generally  regarded  as  a 
significant  element  of  a reliability  program  since  it:  (1)  identifies 
and  helps  the  design  team  to  focus  on  a reliability  requirement 
throughout  systems  development  and  (2),  represents  the  principal  method 
for  verifying  whether  a given  requirement  has  been  achieved  in  the 
design.  Its  positive  role  notwithstanding,  the  procedure  is  often 
not  utilized  in  systems  procurement  mainly  because  of  schedule  demands 
or  cost  considerations.  Part  of  the  problem  arises  from  the  fact  that 
the  principal  demonstration  method  in  use  today,  generally  referred  to 
as  the  "Classical"  method,  tends  to  require  long  tests,  especially  in 
cases  where  the  inherent  reliability  of  an  item  is  large. 

In  contrast  with  the  classical  method  the  Bayes  approach  makes  use 
of  prior  information  about  the  possible  reliability  values  of  the  system 
to  be  tested.  This  information  is  incorporated  into  the  test  design, 
usually  in  the  form  of  a density  function.  The  test  decision  is  now 
based  on  the  prior  data  as  well  as  the  data  generated  during  performance 
cf  the  test.  When  the  prior  distribution  is  amenable  to  a relative 
frequency  interpretation,  dt  can  be  viewed  as  pseudo  test  data  which, 
when  combined  with  actual  data,  is  instrumental  in  effecting  the 
relatively  short  teats  experienced  with  this  method.  In  most  instances, 
however,  development  of  a prior  is  based  on  subjective  information  which 
causes  great  difficulty  in  assessing  the  reasonableness  and  consequences 
of  the  various  ssumptions  made.  This  is  of  considerable  importance 
since  errors  xn  the  estimates  of  the  prior  parameters  are  propagated 
to  the  posterior  estimates,  on  which  decisions  are  based. 

The  Hybrid  method  combines  some  of  the  better  features  of  the  other 
methods.  It  represents  a practical  option  when  the  prior  assignment  is 
considered  to  be  unacceptable  yet  use  of  an  efficient  test  is  indicated. 
Since  the  consumer  protection  is  stated  in  terms  of  the  familiar  classical 
consumer’s  risk,  it  is  independent  cf  the  prior  information.  The  prior, 
however,  still  plays  an  important  role  in  that  it  affects  test  time  and 
the  producer's  risk. 

The  three  rethods  identified  represent  the  range  of  alternatives 
currently  available  in  the  area  if  fixed  time  tests.  Given  a set  of 
test  objectives,  selection  of  suitable  test  plan  requires  knov/ledge  of  the 
characteristics  of  the  different  methods  and  a capability  to  construct  and 
compare  competing  plans.  The  principal  aim  of  this  report  is  to  convey  this 
information  in  simple,  user  oriented  terras. 
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I.  SUMMARY  OF  OTARACTERISTICS  OP  DIFFERENT  TEST  METtiODS 
A.  General 

While  the  risks  of  any  Classical  fixed  tire  test  plan  are  Identical 
in  definition,  this  is  not  true  for  the  Bayesian  n«thod  where,  becattse 
of  lack  of  agreeirent  about  what  is  meant  by  consumer  s and  producer’s  risk, 
thre . different  producer’s  risks  and  two  consumer's  risks  are  utilized  at 
present,  (1),  (3),  (5).  Since  each  producer’s  risk  can  he  combined  with 
a consumer's  risk  to  form  a feasible  test  plan,  it  is  possible  to  construct 
runy  different  Bayes  tests.  In  addition,  each  Raves  risk  can  be  conhlned 
with  a Classical  risk  creating  additional  test  alternatives.  The  full 
range  of  possibilities  can  perhaps  be  most  easily  seen  when  expressed  in 
the  matrix  font:  shown  below: 

MATRIX  OF  POSSIBLE  TEST  METHODS 


1 

2 

3 

1 

ct-B 

a-B 

2 

A-B 

A-B 

A-Bj 

3 

A^-B 

A^-B 

4 

P<Acc)-B 

P(Acc)-B 

P(Acc)-Bj 

the  symbols  denote  the  following:  (A  more  foi*mal  definition  of  the  risk 

terms  is  given  in  the  ne..r  section.) 

a * classical  producer's  risk 

0 ••  classical  consumer  s risk 

A “ Bayes  posterior  producer’s  risk 

B ■ Bayes  posterior  consumer  risk 

A'^  ■ Average  Bayes  producer' s risk 

Bj^  ■ Average  Bayes  consumer’s  risk 

P(Acc)  ■ Probability  of  test  acceptance  (A  test  criterion  used 

in  Bayes  tests  asually  in  lieu  of  the  A or  Aj  criterion) 

ot“S  • designates  the  Classical  cons  timer’s  and  producer's 
risk  combination 

A-B  " designates  the  Hybrid  criteria  utilizing  a Classical 
consumer  risk  and  a Bayesian  producer  risk 
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Partitioning  of  the  matrix  groups  the  array  in  terms  of  the  different 
test  methods.  Thus  element  (1.1)  of  the  matrix  represents  the  Classical 
(ct-B)  method.  Flements  (2,2),  (2,3),  (3,2),  (3,3),  (4,2),  (4,3)  comprise 
the  Eaves  method.  The  other  elements  contain  mixed  criteria  and  are,  bv 
definition.  Hybrid  tests.  Not  all  Hybrid  tests  are  considered  to  be  of 
equal  significance  in  that  those  combinations  appearing  in  the  first 
colust^  (i.e.  , the  ones  utilizing  a classical  consumer's  risk)  are  regarded 
as  more  practical  than  those  with  a classical  producer's  risk  (i.e., 
elements  1^-  and  1,3  of  the  matrix).  This  is  due  to  the  assumption  that 
a Hybrid  plan  is  nreferred  in  cases  where  difficulty  is  experienced  in 
deriving  an  acceptable  prior.  In  these  situations,  it  is  felt,  the 
users  Interest  would  be  better  served  with  a test  where  his  risk  is 
expressed  in  terms  of  « classical  risk.  Plans  which  reverse  this  condi- 
tion (i.e.,  provide  a classical  produced  risk)  were  assumed  to  have 
lesser  prac  ;ical  significance  and  are  not  investigated  further. 

There  '.e  a number  of  assumptions  common  to  all  test  methods.  The 
most  general  is  that  the  interarrival  times  of  failure  ate  considered 
to  be  Independent  and  Identical,  exponentially  distributed,  random 
variables.  The  mean  of  this  distribution  0,  is  the  mean  life.  A direct 
consequence  of  this  assumption  is  that  the  harard  rate  is  constant. 
Independent  of  operating  time.  Another  is  that  the  mean  time  between 
failures  (MTBF)  is  independent  of  the  number  of  failures  observed  and 
equals  the  mean  life.  That  is,  the  mean  time  between  the  (n-1)®^  failure 
and  the  nth  failure  is  the  same  for  all  n (n  = 1,  2,  . .N).  If  the  device 
is  repairable  then  completion  of  a repair  results  in  an  "as  good  as  new" 
condition.  Or,  if  the  device  consists  of  components  which  are  replaced 
upon  failure,  the  device  is  considered  to  be  "as  good  as  new"  after  each 
replacement.  (The  term  'device'  refers  to  that  particular  level  of  a 
system's  hlerarchal  structure  for  which  an  assumption  of  a constant 
hazard  rate  is  considered  reasonable.  Thus  the  term  'device'  could 
signify  a single  component,  a series  of  components  or  a complex  assembly.) 

There  also  exist  commonalities  in  test  methodology.  For  all  method.s 
reliability  demonstration  is  a form  of  hypothesis  test  whose  aim  is 
to  distinguish  whether  6>0l  or  0<6l,  where  0i  is  the  value  of  MTF.F  thnt 
is  to  be  demonstrated.  All  test  procedures  also  call  for  operating  the 
devlc.’.  for  T hours,  repairing  or  replacing  it  upon  failure  (in  cases 
where  repair  is  performed  average  repair  time  (IfTTF)  is  assumed  to  be 
much  smaller  than  the  MTBF)  and  counting,  the  number  of  failures  occurring 
in  time  T.  If  the  number  of  failures  is  less  than  r*,  where  r*  is  the 
allowable  number  of  failures  specified  by  the  test,  the  decision  is  made 
that  0^6j^  (i.e.,  0j^  has  been  demonstrated).  If  the  nunl>er  of  failures 
exceeds  r*  the  decision  is  that  the  O<0i.  The  test  Is  uniquely  deter- 
mined once  T,  r*  are  chosen.  Explicit  specification  of  T,  r*  are 
characteristic  of  a fixed  time  teat.  Tests  where  T and  r*  are  not  fixed, 
i.e.,  sequential  tests,  are  not  considered  here.  In  addition  to  the 
shared  properties,  each  aiethod  clso  has  e number  of  distinct  features 
such  as: 

B.  Classical  Method 

The  probability  of  device  acceptance  depends  on  the  actual  value 
of  0 and  is  denoted  by  P(Acc/0).  The  curve  obtained  when  plotting  this 
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probability  against  0 is  called  the  operating  characteristic  (O.C.) 
curve.  The  O.C.  curve  shows  how  the  chance  of  acceptance  varies  with 
different  possible  values  of  6 and  is  an  important  quality  characteristic 
of  the  test.  Classical  test  design  generally  involves  choosing  a value  of 
0,6o»  si’d  3 value  a (called  producer's  risk)  and  requiring  that  P(Acc/9o 
and  choosing  a value  of  0,6i<Oo  ^nd  a value  g (called  consumer's  risk)  and 
requiring  that  P(Acc/8j)<g.  If  a device  is  accepted  it  is  said  that  mean 
life  0i  has  been  demonstrated  with,  at  least,  confidence  1-g.  The  use  of 
the  parameter  0q,  while  not  a part  of  the  demonstration  requirement,  is 
necessary  to  control  the  shape  of  the  O.C.  curve  for  values  0»9i  , and  thus 
to  protect  the  producer  against  the  use  of  plans  which  have  a iilgh  picb- 
abillty  of  rejecting  devices  which  more  than  meet  the  requirement.  If  all 
devices  tested  are  of  quality  level  the  fraction  c would  be  rejected 
by  the  test.  Similarly,  if  all  devices  have  quality  level  6i,  the  cons-imer 
will  wind  up  with  100%  marginal  devices  even  though  only  B percent  are 
accepted.  This  test  method  assumes  that  all  devices  on  test  have  the 
same,  but  unknown,  MTBF,  and  that  if  future  production  is  accepted  on  the 
basis  of  these  tests,  they  will  also  have  the  same  MTBF.  The  Classical 
method  considers  MTBF  as  an  unknown  parameter  and  no  use  is  made  of  prior 
information  about  the  possible  values  of  MTBF. 

C.  Bayesian  Methods 

Here  prior  information  and  actuJil  data,  each  expressed  in  a 
specified  form,  is  used  in  test  design.  The  mixing  of  the  prior  and  the 
observed  data  is  accomplished  using  Bayes  theorem  and  results  in  a pcsterior 
function  which  reflects  the  impact  of  the  data  on  the  prior.  The  mathematics 
of  the  mixing  process  become  simplified  and  interpretation  of  results 
facilitated  if  the  prior  and  the  test  data  are  chosen  as  conjugate 
functions.  This  leads  to  a posterior  of  the  same  form  as  the  prior  with 
parameters  that  are  additive  constants  of  the  prior  parameters.  In  terms 
of  the  demonstration  i-.’oblem  of  concern  here,  where  MTBF  is  the  figure 
of  merit  and  failure  data  is  generated  in  accordance  with  a Poisson 
process,  the  appropriate  prior  is  the  Inverted  gamnw  density.  As  a con- 
sequence, the  posterior  is  also  an  Inverted  gamma  density  with  parameters 
which  are  the  sum  of  the  prior  parameter  values  and  the  test  data. 

While  the  mechanics  of  the  mixing  process  involve  well  defined  and 
noncontroverslal  operations,  they  can't  be  implemented  without  assign- 
ment of  a prior  distribution.  It  is  this  aspect  oi  the  Bayesian  method 
that  has  been  the  subject  of  great  controversy.  Much  of  it  centers  on 
the  nature  and  interpretations  of  g(9),  the  prior  distribution.  As 
pointed  out  in  reference  (1),  two  models  generally  apply.  The  first 
is  where  0 is  assumed  to  vary  from  experiment  to  experiment,  according 
to  g(0);  that  is,  0 is  assumed  to  be  a random  variable  having  a fixed 
distribution  g(0)  and  the  values  of  the  MTBF  at  different  times  are 
independent  realizations  of  this  random  variable,  which  is  not  directly 
observable.  In  this  model,  the  posterior  distribution  pertaining  to  a 
particular  experiment  cannot  be  used  as  the  prior  for  the  next  experi- 
ment simply  because  this  posterior  is  not  the  one  generating  the  next 
value  of  0.  As  a consequence,  no  information  in  the  form  of  a posterior 
distribution  can  be  carried  over  from  test  to  test.  The  use  of  the 
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posterior  distribution  from  one  test  as  the  prior  for  the  next  builds 
in  the  assumption  that  the  HTBF  at  different  times  is  ideittlcal  and  that 
the  additional  data  pertains  to  the  same  KTBF.  This,  of  course,  is  a 
different  model;  one  which  treats  rfTBF  as  a fixed  but  unknown  constant 
and  its  distribution  g(6)  represents  the  degree  of  belief  associated 
with  the  ptMsible  values  of  6.  iJere  MTBF  can  be  considered  as  a randon 
variable  only  once  and  thereafter  remains  fixed.  This  means  that  addi- 
tional Inforciation  is  gathered  about  the  unknown  parameter  value  as 
more  data  becomes  available,  as  happens  in  sampling  distributions.  Both 
models  /ire  identical  in  that  not  only  the  form  of  the  prior  distribution 
but  thf  specific  distribution  is  assumed  to  be  known  Inltlallv. 

".hese  assumptions  do  not  pertain  to  the  F.npirical  Bayes  method.  Tn 
this  approach,  the  data  accumulated  in  {wrformlng  repetitive  tests  Is 
ased  to  estimate  the  prior  distribution,  which  is  amenable  to  a relative 
freruencv  interpretation.  Investigation  of  the  asymptotic  behavior  of 
the  F.m.plrlcal  Bayes  procedure  in  reference  (2)  indicates  that  after 
ob5.erv.nnce  of  a considerable  number  of  repetitions  the  B.=ye8  risk  for 
the  next  test  is  almost  the  same  as  if  the  prior  were  known.  This  Is 
a very  desirable  property  in  that  nothing  need  he  assumed  about  the 
prior,  but  unfortunately  requires  a long  sequence  of  tests,  of  a 
repetitive  nature,  before  thla  property  is  realized.  The  latter  con- 
straint makes  it  difficult  to  apply  this  method  to  many  reliability 
demonstration  prohletM  and  it  is  therefore  not  discussed  further. 

As  noted  previously,  several  different  sets  of  risk  criteria  are 
utilized  in  the  Bayea  approach.  The  three  commonly  used  seta  which 
arc  also  applied  in  references  (3),  (4),  (5),  are: 

• 

1)  P(0>6o/Reject)  ••  A and  P(0<6j^/ Accept)  ■ B where  A,  B denote 
the  posterior  producer ’s .consumer's risk,  respectively, 

2)  P(Reject/6>GQ)  ■ Aj  and  P(Accept/6<8j)  • where  Aj , Bj 
denote  the  average  Bayes  producer's,  consumer's  risk,  respectively. 

3)  r (Acceptance)  • A2  and  B or  where  A2  is  the  a priori  proba- 
bility of  acceptance  before  the  test  is  conducted.  It  is  used  here  as 
a producer's  criteria  in  lieu  of  A or  Aj. 

The  third  combination  uses  overall  acceptance  rate  for  the  produceT*E 
risk.  It  has  been  suggested  in  references  (3),  (6),  that  this  quantity 
has  tnoi-e  significance  to  a producer  than  either  a or  Ai  .end  therefore 
represents  a roore  pertinent  criterion  than  either  of  the  other  expres- 
sions. Assuming  that  rejected  devices  cost  a producer  money  because  of 
the  need  to  scrap  or  ovcrhaiu  them,  it  does  appear  doubtful  that  be  would 
derive  comfort  from  the  fact  that  only,  say,  10%  of  the  rejected  units 
have  a OiOo  when  his  overall  rejection  rate  is  high.  This  does  suggest 
that  he  may  prefer  to  limit  the  overall  rejection  rate  or  to  control  the 
probability  of  having  good  units  rejected.  To  accomodate  these  goals 
a nuri>er  of  :he  test  plans  utilizing  the  A2-B  and  the  Ay  Bj  criteria 
sets  have  beer,  constructed  and  are  examined  in  the  report. 
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Irrespec.'ive  of  the  particular  set  of  risks  selected,  a Bayes  cest 
cannot  be  formilated  without  use  of  a prior  distribution.  Once  the  prior 
has  been  chosen  the  test  can  be  designed  to  limit  the  risks  to  remain 
within  selected  values  under  the  assumption  that  the  prior  dlscrlbuclon 
is  an  accurate  representation.  This  point  should  be  kept  in  mind  when 
use  of  a Bayes  procedure  la  contemplated.  In  addition,  two  other  potential 
difficulties  are: 

1)  If  the  assumed  prior  is  grossly  in  error,  especially  if  it 
is  overly  optimistic  about  the  capabilities  of  the  device  - this  procedure 
can  be  very  poor  in  detecting  inadequate  devices.  (This  is  bad  for  the 
consumer . ) 

2)  It  may  be  extremely  difficult  to  construct  a prior  which 
accurately  reflects  the  expected  MTBF  capabilities  of  a device  using 
information  gathered  during  its  design  and  development,  or  even  from 
past  data  deemed  "suitable"  for  this  purpose. 

D.  Hybrid  Test  Method 

As  pointed  out,  a serious  problem  mitigating  the  application  of 
Bayes  tests  is  the  uncertainty  associated  with  the  prior  distribution. 

Often  a contractor  has  a prior  distribution  which  puts  a great  deal  of 
weight  on  values  of  Q»Q^  and  the  user  is  unwilling  to  accept  this  prior. 

If  he  does  use  this  prior  for  the  purpose  of  devising  a test  he  finds 
that  I ? has  uncomfortably  large  probabilities  of  accepting  devices  with 
6<9j  and  does  net  share  the  producer's  optimism  -egarding  the  small  chance 
that  such  O's  will  be  encountered.  He  recognises  that  without  having  a 
prior  distribution  he  can  believe  in,  he  cannot  hope  to  achieve  the  aim 
of  risk  B,  the  Bayes  posterior  ccns'imers  risk,  regardless  of  its  potential 
attractiveness.  He  also  realizes  that  using  the  B^^  criterion,  even  with 
reasonably  small  P(Accept/9^8i),  he  can  have  many  poor  devices  on  his 
hands  if  all  devices  tested  are,  in  fact,  of  poor  quality.  He  therefore 
prefers  to  be  protected  in  the  classical  sense,  in  terms  that  he  is  more 
familiar  with.  To  accomodate  this  point  of  view,  a number  of  Hybrid  plans, 
combining  a classical  consumer  and  Bayes  producer  risk,  are  developed  and 
discussed  in  this  report.  As  will  be  seen,  these  plans  have  a number  of 
attractive  features. 
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SECTION  III 


I.  GRAPHIC  PROCEDURE 

A.  General 

When  several  test  alternatives  are  possible,  identification  of 
a preferred  plan  is  usiially  achieved  by  repeated  application  of  a test 
design  procedure.  The  (.haracceristics  of  the  procedure  therefore,  deter- 
mine how  efficiently  th''j  task  can  be  accomplished. 

For  the  classical  test  method,  the  test  selection  problem  is  simple 
since  test  alternatives  are  available  in  tabulated  form  and  appear  in 
publications  such  as  MIL  STD  781B.  Selection  of  an  appropriate  plan 
merely  Involves  scrutiny  of  the  tables  to  identify  that  set  of  parameters 
best  suited  to  meet  the  given  requirements.  Of  course  nor  all  feasible 
combinations  of  test  parameters  are  tabulated.  If  a particular  combination 
not  listed  is  of  interest  the  user  has  a choice  of  either  selecting  a set 
that  comes  closest  to  meeting  his  goal  from  the  available  tabulations  or 
he  can  derive  his  own  plan  by  solving  a set  of  simultaneous  equations 
appropriate  to  the  Classical  test  method.  The  lattei  scheme  requires 
some  additional  effort  but  can  be  readily  accomplished. 

For  the  Bayes  method,  tabulations  are  now  beginning  to  appear, 
reference  (6).  Since  each  plan  must  Include  specification  of  parameter 
values  for  the  prior  distribution,  in  addition  to  the  usual  indices, 
sizeable  tabulations  result.  For  the  inverted  gamma  prior  distribution 
commonly  encountered  parameter  values  vary  over  a 5 to  1 range  for  each 
of  two  parameters.  This  gives  rise  to  a 25  fold  increase  in  the  number  of 
tables  required  compared  to  the  Classical  method (assuming  integer  parameter 
values).  Since  manipulation  of  large  amounts  of  data  tends  to  become  quite 
awkward  and  time  consuming,  this  approach  is  not  considered  to  represent  an 
effective  method  for  test  plan  selection. 

The  graphical  test  design  technique  described  in  this  report  was 
originally  developed  to  facilitate  analysis  of  Bayes  test  plans.  Its 
availability  eliminates  the  need  for  extensive  tabulations  and  removes 
some  of  the  previously  mentioned  limitations  in  regard  to  choice  of  test 
parameter  values.  While  applicable  to  all  test  methods  the  Classical 
method,  because  it  is  the  simplest  and  most  familiar  technique,  is  used 
as  a vehicle  to  explain  the  characteristics  of  this  procedure  and  to 
illustrate  the  manner  in  which  it  can  utilized. 

B.  Classical  Test  Plans 

The  graph  shown  in  figure  1 is  a computer  plot  of  the  risk 
functions  for  the  Classical  method.  The  graph  consists  of  two  sets  of 
curves  labelled  R*A  and  R*B.  The  R*A  set  depicts  the  relationship  between 
producer 's risk  and  normalized  test  times,  T/0o-  A separate  curve  is 
drawn  for  each  of  the  11  values  assigned  to  R*A.  R*A  denotes  the  maxi- 
mum number  of  failures  allowed  in  a test  for  an  accept  decision.  Values 
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of  R*A  are  incremented  in  unit  steps  and  vary  between  zero  and  ten. 
Similarly,  the  R*B  set  describes  the  functional  be>--.i.or  of  consumer's 
risk,  B,  with  norma’  jd  test  time  T/9  . Again  eleven  separate  curves 
are  shown,  each  for  different  R*B  va?ue,  as  indicated  in  that  figure. 

All  the  information  needed  to  formulate,  evaluate  arvi  compare  a 
variety  of  different  fixed  time  Classical  tests  can  be  abstracted  from 
these  curves.  Before  giving  instructions  for  uoin^  this,  it  is  Informative 
to  examine  some  of  the  relationships  shown  in  the  graph.  As  can  be  seen, 
all  consumers  risk  curves  tend  asymtotlcally  towards  zero  with  Increased 
test  time.  Since  ccnsiimer's  risk,  by  definition,  is  the  probability  of 
accepting  marginal  aevlces.  Its  value  is  large  for  small  test  times 
because  even  bad  units  (6<6j^)  will  often  not  fall  in  a relatively  short 
time.  As  test  time  Increases  the  test  becomes  more  discriminatory  in 
that  it  is  able  to  reject  more  bad  units  and  consequently,  the  consumer's 
risk  will  decrease.  For  large  T/0  values  the  test  becomes  rather  severe 
in  the  sense  that  only  a few  failures  are  allowed  over  a ri’iativaly 
long  test  period,  even  for  the  largest  R*B  value  Indicated.  This 
means  that  many  good  units  (9^>6i)  as  well  as  bad  ones,  will  be  rejected 
by  the  test. 

Using  the  definition  of  producer's  risk  and  applying  similar  reasoning, 
it  is  expected  that  for  small  test  times  a will  be  small  since  few  units, 
good  or  bad,  will  be  rejected.  Conversely,  for  larger  test  times  a will 
be  large  due  to  the  severity  of  the  tesr.  When  T/9  Is  kept  constant,  a 
can  be  seen  to  vary  Inversely  with  R*.A.  This  resui?  can  be  explained  by 
noting  that  conditions  for  acceptance  have  been  relaxed  when  the  value 
of  R*A  is  increased,  while  T/9  remains  fixed.  Consequently  the  proportion 
of  rejected  units  will  be  smalJer  and  therefore  the  subset  consisting  of 
the  fraction  of  rejected  units  that  are  good  will  also  be  smaller. 

Depicting  the  pertinent  functions  in  graphical  form  permits  ready  obser- 
vation of  their  behavior  and  thus  promotes  an  understanding  of  important 
relationships. 

As  pointed  out  previously,  the  graphs  of  figure  1 can  be  used  as  a 
simple,  accurate,  yet  flexible  test  design  tool.  While  the  application 
sequence  will  vary  in  accordance  with  the  specific  requirements  of  a 
given  problem,  the  casvs  generally  encountered  can  be  classified  into 
3 groups.  In  the  following  discussion,  sample  problems  of  each  group 
are  postulated  together  with  a step  by  step  explanation  of  how  the 
respective  solutions  may  be  obtained. 

i.  Plans  having  equal  risks  and  a constraint  on  the  common  risk 
value . 


Example  1:  Develop  rest  plans  having  equal  a and  G values.  For 

tne  purpose  of  this  example  limit  a,  6<20%. 

The  values  of  T/0o,  and  the  maximum  allowable  number  of  failures,  r* 
needed  to  completely  specify  th.is  test  can  be  obtained  from  the  points 
of  intersection  of  equal  valued  R*A  and  R*E  curves.  Since  valid  test 
conditions  require  a single  value  for  the  allowable  number  of  failures, 
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R*A  «u8t  equal  R*B.  The  point  of  intersection  of  the  R*A  *=  R*B  * 10  curve 
gives  a value  of  12. 6X  for  equal  a,  6-  The  corresponding  tect  time  can  be 
determined  by  drawing  a vertical  line  from  the  point  of  intersection  to  the 
horizontal  (x)  axis.  The  value  obtained  is  approximately  7.4  0o hours.  If 
five  failures  are  allowed,  test  time  is  decreased  to  approximately  4.0  0o 
hours  but  the  risks  increase  to  20.42.  The  latter  plan  represents  the 
lower  limit  for  allowable  number  of  failures  since  any  further  reduction 
causes  the  maximum  risk  values  specified  to  be  exceeded.  Therefore,  only 
test  plans  with  r*>5  satisfy  the  given  requirement.  The  graphs  clearly 
show  the  existing  trend:  an  increase  in  r*  results  in  decreased  risks  but 

is  also  accompanied  by  longer  test  times. 


2.  Plans  having  unequal  risks  and  separate  constraints  on  each  risk. 

Example  2:  Devise  a test  with  a and  6 having  specified  but  unequal 

values.  For  this  example  assume  that  an  a of  182  and  a 6 of  approxi- 
mately 132  is  desired. 

The  procedure  for  obtaining  the  required  plan  is  only  slightly  more 
complicated  than  that  of  the  pre\'lous  example.  Again,  it  should  be 
observed  that  establishment  of  viable  test  conditions  requires  that  R*A 
equals  R*B  equals  r*,  and  that  both  the  allowable  number  of  failures  .,nd 
test  time  have  to  be  single  valued.  Within  these  restrictions,  the 
needed  T/9  and  r*  values  are  obtained  by  drawing  a horizontal  line  through 
the  a-18  point  on  the  Y axis.  The  line  should  intersect  all  R*A  curves. 
Pick  an  R*A  curve,  say  R*A*=7,  and  draw  a vertical  line  from  that  point 
of  intersection  to  the  corresponding  R*B  curve  (R*B-7,  in  this  instance). 
The  latter  point  of  intersection  provides  the  S value  for  this  test, 
which  is  approximately  162;  when  the  vertical  line  is  extended  to  the 
abscissa  the  corresponding  test  time,  approximately  5.4  6 hours,  can  be 
obtained.  Since  the  consumer's  risk  (16%)  exceeds  the  des?rcd  value  (132) 
the  procedure  is  repeated,  this  time  choosing  the  R*A=8  cuf'e.  An  a of 
182  on  this  curve  delineates  a 3 of  12.02  and  a corresponding  test  time 
of  6.3  6 hours.  Since  requirements  are  met,  the  latter  plan  is  accept- 
able. In  instances  where  some  flexibility  exists  in  the  statement  of 
test  requirements,  the  graph  can  be  utilized  in  other  ways.  For  example, 
the  data  indicates  that  a 32  reduction  in  3 can  be  realized  by  allowing 
test  time  to  increase  by  a normalized  unit  (T/6  =1).  Whether  this 
represents  an  acceptable  tradeoff  depends  on  the  constraints  of  the 
specific  problem  addressed.  The  example,  however,  illustrates  another 
potential  area  of  application  for  this  procedure. 

3.  Plans  having  a time  constraint  and  (possibly)  a single  risk 
constraint. 

Example  3:  Develop  test  plans  where  test  length  does  not  exceed 

5.0  9 hours. 

o 

All  feasible  plans  are  delineated  by  drawing  a vertical  line 
through  the  T/9^=5.0  point  and  are  situated  on  or  to  the  left  of  that  line. 
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Ciiooslnp  the  K*A“]0  curve,  an  a of  approxlr>ately  1.4"  is  ootalneci.  !iov- 
ever,  tise  cor'ospomiinp  S cannot  be  obtalnoi!  directly  fror  the  praph 
4lhcc  Its  valu*  exceeds  307,  By  lettlnp  a tnl-.e  on  larp.er  values  the 
correspomilnp  0's  becore  smaller.  For  o equat  to  ] 3"  and  iislnp  Che 
previous  test  tine  of  T/6g»5.0,  t'ne  appropriate  ? Is  ?2".  This  value 
is  obtained  fron  the  poi»'t  of  intersection  of  the  T/6^=5  line  and  t’lr 
R*B“7  curve.  Ttie  selection  of  the  R*!’.*=7  curve  is  based  on  the  reouire- 
•■ant  that  R*;j  nust  equal  R*A.  (R*/i=7  had  previously  bec'i  <’efinei'  by 
choice  of  ciie  a and  T/6q  values.)  Other  solutions  can  - e ol  tair.ed  !v 
choosing  another  feasible-  test  tire  and  folloKinr  a ro-:tine  slrilar  to 
that  descrlbeil. 

/»t  this  point  It  wouJc!  perhaps  be  instructive  to  shoe.-  hoi:  reaiiily 
the  test  conditions  for  sore  fixed  tire  pl.ans  of  ’SnSTi'  7°!''  ran  be 
re.produced  slnp  the  graphical  procedure.  For  exarple,  test  plan 
in  tal'le  4,  pace  1 specifies  a producer’s  rlsl  of  ]0*-,  requires 

6.2  hours  of  normalize. 1 test  tire  an'  allcn-s  ‘■allurf's;  test  ^lan  '"I  in 
the  sane  table  is  a plan  having  an  a* 3=20';  a T/6o= bo-irs  and  allows 
5 f.ailures  for  acceptance,  witli  the  aid  of  fi«-ure  1 an-'  use  of  any  2 
of  the  4 quantifies  specified  for  each  plan,  t'se  other  -.-alucs  can  he 
readily  v»'riii£'.  Th.e  reason  for  this  particular  rhoifc  of  ’tll.RTp  7P1P, 
plans  is  that  the  vrluos  of  the  discririnat  ion  r.atio,  .an  ’ the  allow- 
able n-ir!  cr  of  failures,  r*,  are  identic.al  to  tiiose  used  In  constructing 
this  graph,  ho'-cver,  all  plans  shown  in  that  t.nble  can  be  derived  graph- 
ically if  the  curves  arc  redrawn  incorporating  the  appropriate  jiaraneter 
adjustnents. 

C.  Application  to  Bayes  Test  Pl.ans 


Test  plans  utilizing  either  a corbination  of  Classical  and 
Bayes  (Hybrid)  or  both  Bayes  criteria  cannot  l>e  forrulate-*  without  sore 
prior  distribution.  For  reasons  of  convenience  and  because  of  Its  vide 
usage,  Che  inverted  g,ar--a  distribution*  served  as  the  prior  distribution 
for  all  Bayesian  and  Hybrid  plans  described  here.  This  density  function 
can  be  expressed  as:  /^\ 


where  6 denote  the  shape  and  scale  parareter  of  the  distribution. 

The  rean  cf  this  distribotlon,  E(6),  equals  </{o-l)  for  (^>1  and  r(if) 
is  the  garsna  function  of 

* Results  from  a change  of  variables  such  that  the  new  variable  is  the 
inverse  of  the  gamma  distributed  random  variable. 

The  table  shown  on  page  III-6  summarizes  the  characteristics  of  the 
Bayes  functions  graphed  and  establishes  a frame  of  reference  for  this 
discussion- 
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Dlscrlninaclon 


I? 

fc 

B 


Criteria 

Set 

Paracxitcr  Values 
shape  scale 

parameter  paraneter 

Expected 

Value 

n(6> 

A-B 

1 

150 

75 

A-Bj 

1 

3 50 

75 

Af-Bi 

3 

150 

75 

P(Acc)-3 

3 

200 

100 

f.raph  is 
shown 

ratio  In  flpurc 

K 

2 2 

2 1 

2 4 

2 5 


Thp  shape  of  the  Hayes  risk  functions,  as  can  be  observed  fror  the 
graphs,  reser.hles  the  Classical  a,  3 curves,  especially  for  tht  Aj-Bj 
criteria  set.  Since  the  Classical  and  the  Aj-Bj  risks  have  sirli.ar 
definitions,  their  likeness  is  not  too  surprising-  The  Bayes  risks,  how- 
ever, are  expressed  in  terris  of  a probability  which  is  averaged  with 
respect  to  a region  of  the  prior  clstribuiion  (i.c. , the  conditional 
probability  of  rejection  is  defined  for  values  of  6>ep)whereas  the 
classical  risks  are  specified  in  terras  of  point  values  for  S. 

This  sinilarity  also  carries  over,  to  sojtc  extent,  to  the  other  Bayes 
plans.  A point  of  significant  difference  is  that  the  Classical  consucer s 
risk  curve  will  always  start  at  one  (i.e.  have  a value  of  one  for  zero 
test  tiiae)  and  will  asyciptotlcally  approach  zero  for  large  test  tine. 

The  Bayes  consur.ei%  risk  will  generally  not  do  this.  The  Initial  value 
is  detennined  by  the  area  in  the  prior  density  bounded  by  0<6<8j^.  Alter- 
natively, the  Bayes  producer‘s  risk  curves  will  generally  not  converge  to 
an  as3rntote  of  1 for  large  test  tire  as  is  the  case  with  the  Classical 
producer’s  risk.  The  llr.itlng  value  reached  depends  on  the  area  sepnent 
in  the  prior  density  defined  by  the  Units  *.  These  features  and 

their  effect  on  test  plan  characteristics  are  exarined  in  r^ore  detail  in 
the  next  section. 

Graphs  of  criteria  set  P(Acc)-B,  shown  in  figure  3,  exhibit  the 
greatest  difference  compared  to  Classical  plans.  The  usual  approach  to 
test  design  using  this  criteria  set  is  to  initially  select  values  for 
P(Acc)  and  the  consiaaer’fl  risk  and  then  calculate  the  other  parancters. 

Since  P(Acc)  indicates  the  probability  of  successfully  passing  the  test, 
it  is  in  the  producer**  interest  to  specify  a large  value  for  this  quantity, 
(i.e.  within  the  litsiting  value  of  one).  Of  course  the  P(Acc)  value 
should  not  be  picked  arbitrarily,  but  based  on  the  prior  parauaiter  esti- 
isates,  choice  r<f  allcwable  maaber  of  failures,  r*,  and  test  time,  T. 
Increasing  r*  Increases  P(Acc)  but  it  also  increases  T,  for  a given  level 
of  o)nsuoer*srlsk.  Since  it  is  generally  desirab'e  to  keep  T small,  an 
inflated  P(Acc)  estimate  is  likely  to  reflect  an  expected  test  advantage. 
Since  E{6)  is  coeluted  from  the  same  parameter  estimates  it  will  also 
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be  affected;  the  iirection  of  chanpe  is  such  that  an  increase  in  r(Acc) 
will  also  increase  n(6).  Test  plans  with  large  P(Acc)  and  r(8)  generally 
lead  to  very  short  tests  as  is  evidenced  by  soc-e  plans  appearing  in 
reference  (6).  The  problec  with  short  tests  is  that  the  observed  data 
generally  is  inadequate  to  influence  the  test  decision. 

The  use  of  these  plans  is  therefore  not  reconrended.  P(Arc)  can, 
however,  be  constructively  used  in  another  canner;  It  can  be  er;ployed  as 
a technique  for  predicting,  the  reliability  status  of  a product  in  the 
various  stages  of  its  devoloptaent  and  thus  help  to  flag  problers  that 
ray  require  corrective  action. 

The  application  of  the  graphical  procedure  to  formilate  r.ayes  tests 
generally  follows  along  the  lines  previously  discussed-  Powever,  several 
additional  exai^les.  with  arbitrarily  chosen  parareters,  will  be  presented 
to  further  illu'trate  this  technique- 

Exanple  (4):  Assune  a Bayesian  test,  ssing  ciltcria  A-B,  is  to  be 

developed  with  the  requirenent  that  it  ' not  exceed  2-0  test  units, 
netemlne  all  pertinent  test  parareters. 

A vertical  line  drawn  through  the  T/6o“2  point  on  the  abscissa 
delineates,  to  the  left  of  that  line,  all  practical  test  possibilities. 
rnoi«e  of  r**l  for  the  allowable  nurher  of  failures  leads  to  n test  plan 
with  a producer.*"  risk.  A,  of  approxicatcly  9.57;  a consurers  risk,  B,  of 
approxicwitely  6*  for  T/flo*2.  Reducing  test  ticse  to  T/9o**1.75  results  in 
a plan  where  A»B»67  when  r*»l.  Further  reduction  in  test  tire  to  T/6o*l 
results  in  a test  condition  where  A»D®127  and  r*=0. 

The  rcqulreiaent  that  T/0q  not  exceed  2 units  is  ret  by  all  plans. 

The  final  selection  process  therefore  ir»ost  include,  other  considerations. 
However,  the  ease  with  which  viable  alternatives  can  he  identified  throueh 
use  of  this  procedure  is  worth  noting. 

Exanple  (5):  /Vssune  that  a given  test  probien  does  not  dictate 

use  of  a specific  Bayes  plan  and  that  all  feasible  Bayes  tests  will  be 
considered  for  test  plan  selection.  Specifically  what  i.s  desired  Is  a 
plan  having  equal  risks,  not  to  exceed  10%,  and  ninimu?  test  tine. 

The  following,  plans  using  the  A-T.  criteria  depicted  in  figure  ? will 
reet  the  requirerents: 

a)  ■^/Ojj-l.TS  r**l ; A“E**8.67 

h)  T/9o-2.5  r*»2;  A-r-6.5r 

There  are  several  other  po.ssibilities  but  they  are  not  practical 
because  of  exccssiw  test  Icngti's. 

Soise  viable  candidates  using  the  Aj-Bj  criteria  set,  shown  in 
figure.  4 arc: 

a)  T/eo-1.75  r*-l;  Ai-Bi“13.6r 

b)  T/6o*2.5  r*»3;  Aj-B 1-10.57 
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The  feasible  plans  of  the  A-Di  criteria  set,  shown  in  flpure  3,  are: 

a)  T/0O-1.5  r*-l;  A-Bi-7.87 

h)  T/0O-2.25  r*-2;  A-Bi-6/ 

A coaparison  of  plans  with  approximately  equal  test  time  (i.e.,  the 
coaparison  is  anong  plans  (a)  or  plans  (b)  in  the  different  test  cate- 
gories) indicates  plans  using  the  A-B  or  the  A-Bj  criteria  are  preferable 
to  the  Ai-Bi  plans,  since  they  offer  lower  risks.  It  is  to  be  er-phasized, 
however,  that  the  purpose  of  this  example  Is  to  Illustrate  use  of  a 
technique;  not  to  suggest  that  different  test  criteria  can  surr-arlly  be 
subjected  to  a trade-off  study;  or  to  lrq>ly  that  selection  of  a prefer- 
able plan  be  based  solely  on  minirun  risk  or  ciinlmii5  test  tire.  A 
course  of  action  for  these  situations  is  proposed  in  a subsequent  section 
of  this  report. 

Lxai^ple  (f>) : Devlae  a Bayesian  test  plan  which  has  an  a priori 

probability  of  acceptance,  P(Acc),  of  907. 

Draw  a horizontal  line  through  the  P(Acc)*.9  point  on  the  Y Axis 
of  the  curves  shown  in  figure  5.  The  intersection  of  this  line  with  I 

each  r*  curve  (tns  r**0  curve  is  not  tised  because  the  point  of  intersec- 
tion Is  too  close  to  the  tine  origin)  identifies  potential  candidates  In 
tert:s  of  tost  time,  P(Acc)  and  r*.  To  obtain  the  corresponding 
value  of  constc«r's  risk  construct  a vertical  line  from  a given  point 
of  intersection  to  the  consumer's  risk  curve  having  the  sane  r*  value.  Thus, 
for  the  r*»l  curve,  a B value  of  .21  and  a test  tine  of  .3  6o  imits  Is 
obtained.  For  the  r**2  curve,  the  corresponding  value  is  B»20,  T».7  0q 
units;  the  r*-6  curve  yields  a B of  .16  and  T*2.1  6q  units.  Tnc  rela- 
tively siiort  test  tines  indicated  for  these  plans,  even  for  the  r*“6  plan, 
tends  to  cenfim  previous  remarks  concerning  the  expected  shortcom.lnps 
of  these  tests. 

n.  Hybrid  Test  Plans 

The  outstanding  feature  of  Hybrid  plans  is  that  the  consunet's 
risk  is  not  affected  by  the  prior  distribution.  This  property  makes 
utilization  of  these  plans  especially  attractive  in  cases  where  difficulty 
is  experienced  in  the  assignsent  of  an  appropriate  prior  cr  when  assump- 
tions underlying  its  foroulaticn  are  tenuous.  As  a test  method  Hybrid 
plans  offer  a nuzjier  of  advantages  over  the  Classical  approach,  stxvj  of 
which  will  be  discussed  in  the  next  section.  However  when  viewed  in 
terms  of  thetr  consuner/producer  risk  curves  they  offer  nothing  new. 

The  Classical  S is  the  consuc«r  s risk  function  used  in  all  the  graphs 
depicting  this  test  method,  while  several  diffe'^ent  Bayes  ztiteria  are 
used  for  the  producer’s  risk.  Figure  6 denotes  the  .*-3  ccm.'iinatlon; 
figure  7 the  Aj^-3,  and  figure  8,  the  P(Acc)-S  set.  A priori  pararseter 
values  are  identical  to  those  previously  used,  i.e.,  d'150,  i*3. 

In  addition  to  the  obvious  application  as  a device  for  test  plan 
developtsent,  the  graphs  have  utility  In  ocher  respects  as  well.  For 
example,  when  evaluating  alternative  approaches  it  may  be  of  interest 
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to  detemine  the  value  of  the  Classical  consumer's  risk  that  corresponds 
to  the  consumer's  risk  of  a given  Bayes  plan.  To  illustrate  how  this  may 
be  accomplished,  the  three  different  Bayes  plans  of  example  I',  with  the 
following  properties,  are  utilized; 

1)  T/ 00=2.0;  r*=l;  A«9.5'<;  B=67 

2)  T/8o-2.75;  r*=l;  A*=n=8^6% 

3)  T/0O-1.O;  r*-0;  A-B=12r< 

Example  (7):  The  classical  6's  that  conform  to  the  6 risks  listed 

above  can  be  ascertained  with  the  aid  of  the  graph  of  the  Hybrid  method 
contained  in  figure  6.  (Conformance  is  established  in  the  sense  that  this 
value  of  6 determines  a Hybrid  plan  which  is  identical  to  the  original  Bayes 
plan  in  all  other  test  parameter  values.)  Any  combination  of  two  of  the 
three  quantities  (T/6o,  r*.  A)  listed  above  can  serve  to  locate  the  appro- 
priate value.  In  the  first  plan,  using  T/0o*2.O  and  r*=l,  a 6 value 
of  0.5%  is  obtained.  For  the  other  plans,  B's  equal  to  13.5/  and  13.0% 
were  obtained  by  following  the  same  procedure. 

Example  (3):  Perhaps  another  interesting  example  is  one  involving 

comparison  of  plans  using  different  criteria.  Specifically  what  is  wanted 
is  a measure  of  the  differences  between  plans  with  A-6  and  Aj-6  criteria 
sets. 


Maintaining  the  consumer's  risks  fixed  at  a valiie  of  10%,  one  possible 
plan,  using  risks  Ai-B  has  a producer’s  risk  of  217  for  T/6o*3.3  and  r*=3. 
Another  plan  using  the  A-6  criteria,  again  with  consumer's  risk  of  107, 
has  a producer's  risk  of  12.47  and  requires  only  T/0o=1*15  test  units  for 
r*=0.  If  the  test  lengths  are  made  equal  by  adjusting  the  tire  of  the 
A-6  plan,  this  plan  will  now  have  a producer's  risk  of  approximately  5.P7 
while  the  consumer's  risk  remains  at  10%. 

An  equal  risk  plan  using  criteria  Ai~6  requires  T/0o=5.25  units  of 
time  allows  six  failures  and  has  risks  of  A“6=10.47.  For  criteria 
combination  A-0,  a comparable  plan  in  terms  of  the  risks,  has  A=8=9.57 
and  can  be  performed  in  2 Go  test  hours,  allowing  one  failure.  The 
difference  in  test  time,  while  substantial  iicre,  depends  on  the  values 
assigned  to  the  a priori  parameters.  Again,  these  examples  are  not 
intended  to  imply  that  one  set  of  criteria  is  readily  interchangeable 
with  other  sets;  rather  it  is  suggested  that  a careful  review  of  candi- 
date criteria  sets,  to  determine  whether  they  represent  meaningful  figures 
of  merit  in  a given  problem  situation,  r^recede  a comparison  of  the  type 
outlined  above. 

In  tl'T  previous  section  it  was  rccommendod  that  P(Acc)  not  be  used 
as  a test  cricerion  but  as  a means  of  deciding  whether  the  device  to  be 
tested  is  ready  for  ter-t  or  in  need  of  rework.  The  graph  shown  in 
figure  8 can  be  used  to  calculate  the  value  of  P(Acc)  on  which  this 
decision  can  be  based.  To  implement  this  requires  that  any  two  of  the 
four  quantities  that  define  the  test  (i.e.;  T/0Q;r*,  consumer's  risk, 
producer s risk)  be  available  in  numerical  form;  the  appropriate  P(Acc) 
value  can  then  be  determined  from  the  graph  of  figure  P. 
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SECTION  IV 

I.  CHANGES  IN  TEST  PLANS  AS  A FUNCTION  OF  CHANGES  IN  PARAMETER  VALUES. 

A.  General 

The  aspect  of  Bayesian  tests  that  is  ot  much  practical  concern 
is  the  influence  of  the  prior  distribution  on  the  test.  Since  it 
generally  is  difficult  to  arrive  at  reliable  estimates  for  the  prior 
parameters,  it  is  of  considerable  interest  to  determine,  for  example, 
how  errors  in  these  estimates  change  preselected  risks  or  otherwise 
affect  test  properties.  Although  there  are  a number  of  articles  in 
the  literature  that  attempt  to  deal  with  this  problem,  conclusions 
differ  and  seem  to  depend  on  the  variables  analyzed. 

For  the  inverted  gamma  prior  distribution,  this  study  has  found 
that  the  Bayes  risks  are  quite  sensitive  to  the  expected  value  of  this 
distribution,  E (6).  In  fact  the  difference  between  E(6)  and  the  re- 
quired MTBF,  00,  constitutes  a useful  easure  in  the  analysis  of  this 
problem.  It  should  be  recalled  that  E(6)  represents  the  value  of  MTBF 
derived,  either  empirically  or  on  a personal  probability  basis,  prior 
to  conducting  the  test.  9o,  on  the  other  hand,  is  the  value  of  MTBF 
that  reflects  the  requirements  of  the  intended  application.  The 

demonstration  statement  is  usually  posed  in  terms  of  another  value  of  i 

MTBF,  Oj,  denoting  a minimum  acceptable  MTBF.  When  E(0)  is  large  i 

compared  to  6o,  for  a given  0i,  there  is  favorable  expectation  of  i 

successfully  completing  the  test.  This  condition  is  designated  as  the  r 

optimistic  case,  or  one  with  an  optimistic  prior,  and  is  in  contrast  | 

with  the  pessimistic  case,  where  E(6)<6o.  The  latter  case  reflects  a i 

situation  where  the  pre-test  MTBF  estimate  is  less  than  the  value  j 

specified  for  the  test.  Consequently,  the  chance  of  successfully 
passing  the  test  is  small,  assuming  the  estimate  is  reasonably  repre- 
sentative. In  the  optimistic  case,  a large  area  segment  in  the  prior  | 

distribution  is  generated  by  values  of  9 between  the  limits  Go^0£  <»  ; 
whereas  in  the  pessimistic  case,  values  of  9 within  the  range  O£9£0i, 
will  give  rise  to  a large  area.  That  is,  large  left  or  right  tail  aieas 
result  when  the  product  is  estimated  to  be  either  very  bad  or  very  good 
compared  to  the  required  value,  Oq. 

The  approach  used  to  study  the  impact  of  the  prior  consisted  of 
constructing  graphs  of  selected  Bayes  and  Hybrid  risk  functions  for 
various  values  of  expected  MTBF,  E(9),  and  to  make  inferences  concerning 
the  differences  observed.  Specifically,  criteria  sets  A-B,  A-6,Ai-Bi 
and  Ai-6  were  selected  for  this  analysis.  Five  graphs  were  plotted  per 
criteria  set,  each  with  a different  value  of  the  scale  parameter,  6,  as 
follows:  6=  100,  150,  200,  300,  400.  As  a consequence,  E(0)  took  on 
values  of  50,  75,  100,150  and  200,  respectively.  0q  was  chosen  to 
equal  100,  and  ©i  was  50.  This  assignment  permitted  examination  of 
both  optimistic  and  pessimistic  priors. 
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B.  Effect  on  Bayes  Plans 

Figures  9-13  are  plots  of  the  risk  functions  of  criteria  set 
A-B  vs  time.  In  the  initial  graph  E(9)  equals  50  and  since  9o=100,  it 
exemplifies  the  pessimistic  case.  In  the  succeeding  graphs  E(9)  takes 
on  increasingly  more  optimistic  values  until  a value  of  200  is  reached 
as  shovm  in  Figure  13.  The  different  initial  and  final  values  of  the 
risk  curves,  a condition  previously  noted,  are  readily  observable  in 
these  graphs.  As  explained,  these  values  represent  probabilities  which 
are  derived  from  the  prior  density;  moreover,  their  relative  magni- 
tudes can  be  inferred  directly  from  the  classification  assigned  to  the 
prior  (i.e.,  a prior  is  classified  as  pessimistic  or  optimistic  relative 
to  the  test  requirement.  Go-  Also,  a pessimistic  prior  implies  a large 
left  tail  area;  an  optimistic  prior  a large  right  tall  area.) 

The  relationship  between  the  size  of  the  tail  areas  and  the 
limiting  values  assumed  by  the  risk  functions  follows  directly  from  the 
definition  oj.  these  risks.  For  example,  by  definition,  the  Bayes  pro- 
ducer risk  is  a conditional  probability  indicating  the  fraction 
of  rejected  units  that  are  good  (good  means  that  9^0o).  For  long  tests, 
this  conditional  probability  will  equal  P(G>.9o),  the  probability  of 
having  good  units  on  test.  However,  P(9^9o)  is  also  the  right  tail  area 
of  the  prior  distribution.  Therefore,  an  optimistic  prior  will  give  rise 
to  a large  asymptotic  value  of  producer s risk.  Conversely,  a small 
limiting  value  of  producers  risk  is  caused  by  a pessimistic  prior. 
Moreover,  as  this  function  increases  monotonically  with  time  the  limiting 
value  also  represents  the  maximum  value. 

Similarly,  the  consumer's  ri.«k  which  is  defined  in  terms  of  the 
percentage  of  accepted  items  that  are  bad  (i.e.,  have  a 0£^9i)  will, 
as  test  time  approaches  zero,  equal  the  percentage  of  bad  items  on  test, 
P(0^9j).  For  a pessimistic  prior  this  will  be  a large  value.  As  more 
optimistic  cases  are  encountered,  by  increasing  E(9),  this  magnitude 
will  decrease.  Finally,  when  the  expected  MTBF  is  twice  the  required 
value  (i.e.,  E(0)=29o),  the  situation  depicted  in  Figure  13,  the  con- 
sumer’s risk  curves  will  not  only  have  a small  Initial  value  but  will  be 
essentially  independent  of  test  time  over  an  extended  portion  of  total 
test  time.  Since  these  curves  decrease  monotonically  with  time,  the 
initial  value  also  represents  the  maximum  value.  Thus,  with  an  optimistic 
prior  it  is  possible  to  formulate  test  plans  having,  both,  a very  small 
consumer's  risk  and  a very  short  test  time,  as  exemplified  by  plans  that 
may  be  developed  from  the  graph  of  Figure  13.  The  basic  problem  with 
this  type  of  test  is  its  almost  total  dependence  on  the  prior  MTBF 
assignment.  IJhe:.  it  can  be  established  that  the  prior  does  accurately 
reflect  the  MTBF  characteristic  of  concern,  these  plans  may  be  attractive 
in  terms  of  the  cost  savings  they  offer.  Difficulties  arise,  however, 
when  this  is  not  the  case.  The  tabulated  data  sho'.-'n  below  indicates  how 
rapidly  the  risks  change  as  a function  of  the  prior  parameter  values. 

The  data  was  derived  from  the  curves  of  Figures  9-13  using  a normalized 
test  time  T/0o=1.5  units  and  r*  = 1 for  all  data  points  shown. 
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E(0) 

6 

A 

B 

50 

100 

.028 

.22 

75 

150 

.078 

.114 

100 

200 

.15 

.06 

150 

300 

.3  (estim/ited) 

.014 

200 

400 

(cannot  be 
determined 
from  the 
graph) 

.002 

As  indicated  in  this  table,  a 4 to  1 variation  in  E(9),  or  6 j, causes 
approximately  a 100  to  1 change  in  the  consumers  risk  and  a correspond- 
ingly large  change  in  the  producer's  risk.  While  the  selection  of  the 
T/00,  r*  values  used  in  this  example  may  seem  arbitrary,  the  results  are 
indicative  of  the  magnitude  of  the  risk  excursions  that  will  genercilly 
be  encountered.  This  may  be  verified,  with  the  aid  of  the  graphs,  in 
the  following  manner:  Select  an  arbitrary  set  of  risk  curves  (i.e.,  one 

consumers  risk  curve  and  a producers  risk  curve  with  the  same  r*  value) 
and  examine  the  change  in  the  risk  functions  as  E(6)  is  varied.  Th.'s  is 
best  accomplished  by  picking  a number  of  points  on  the  time  axis  and 
determining  the  risk  values  corresponding  to  these  points  for  each  graph 
shown  in  Figures  9-13.  Review  of  this  data  will  permit  determination  that, 
for  a given  r*,  as  T/0q  is  increased  the  variation  in  E(6)  with  consumer's 
risk  will  decrease,  while  the  opposite  is  true  for  the  producer's  risk. 
However,  sluce  short  tests  have  greater  practicality  and  are  therefore 
utilized  more  frequently,  the  previous  conclusion  is  generally  applicable. 

Bayes  plans  utilizing  criteria  set  Aj-Bj  are  shown  in  Figures  14  to 
18.  The  characteristics  of  this  set  were  not  examined  in  detail  although 
it  was  noted  that  changes  in  parameter  values  have  a much  smaller  effect 
on  the  risk  functions  and  chat  the  different  initial  and  final  values 
previously  observed  were  absent,  at  least  over  the  range  of  values  plotted 
in  the  graphs. 

C.  r.ffect  on  Hybrid  Plans 

Figures  19-28  illustrate  the  impact  of  parameter  changes  on 
Hybrid  plans.  As  can  be  seen,  the  consumer's  risk  function  is  not  affected 
by  these  changes  while  the  variation  in  the  producer's  risk  curves  is 
identical  to  those  displayed  in  previously  presented  graphs,  appearing 
in  Figures  9-13.  The  fact  that  consumer's  risk  is  independent  of  the  prior 
distribution  precludes  Implementation  of  plans  with,  boch,  a small  con- 
sumer 6 risk  and  a short  test  time. 
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In  addition  to  providing  good  consumer  protection.  Hybrid  tests 
offer  another  advantage  in  that  they  generally  can  be  completed  within  a 
reasonable  time  frame.  However,  a quantitative  comparison  between 
Classical  and  Hybrid  plans,  necessitates  establishment  of  a specific 
set  of  ground  iules.  Based  largely  on  the  thought  that  a Bayes  and  a 
classical  producer's  risk  are  alike  in  the  sense  that  both  represent 
decision  errors  whose  magnitude  is  to  be  controlled  by  the  test,  an 
equal  value  assignment  was  made.  Equal  values  were  also  assumed  for  the 
consumer's  risks,  because  of  the  resulting  simplification  in  the  data. 

The  pertinent  data  may  be  arranged  as  shora  below: 


Choice 

of  a = 

A = 

13%  and 

6 = 

13% 

provides 

the  following  plans: 

K=2 

9o=l00 

E(G) 

r* 

T/Oo 

8 

A 

B 

Comment 

Classical 

- 

10 

7.4 

13% 

- 

13% 

Hybrid 

50 

No  plans  are  possible 
for  these  risks 

75 

0 

1.0 

— 

12 

12 

100 

2 

2.5 

- 

12. 

,5  12.5 

200 

8 

6.5 

- 

13 

13 

These  are  extrapolated 

results; 

r*  values  have  to  be 
increased  beyond  the 
number  shown  in  the 
graph  to  get  definitive 
points. 

Choice  ofA  = ct=B=5.5%  results  in  the  following  plans: 


K=2 

0o=100 

E(9) 

r* 

T/6o 

6 

A 

6 

Comment 

Classical 

- 

14 

10 

5.5 

- 

5.5 

Extrapolated  value 

Hybrid 

50 

0 

1.5 

- 

5.5 

5.5 

75 

4 

4.5 

- 

5.5 

5.5 

100 

7 

6 

- 

5.5 

5.5 

Extrapolated  value; r* 
must  be  greater  than 

150 

6 to  get  specific  test 
points. 

200 
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The  T/9o  column  Indicates  the  amount  of  test  time  required  for 
each  plan. 

The  tabulation  shows  that  Hybrid  plans  require  less  test  time  than 
their  classical  equivalents  (equivalency  is  in  terms  of  the  risk  values), 
with  the  largest  savings  in  test  time  occurring  for  Hybrid  plans  with 
pessimistic  priors. 

D.  Effect  on  the  Discrimination  Ratio. 

To  this  point,  a discrimination  ratio,  K of  two  has  been  used 
in  all  the  graphs  presented.  The  fact  that  test  properties  change  as 
the  K ratio  is  changed  is  a well  documented  result  for  the  Classical 
method  and  applies  to  the  Bayesian  and  Hybrid  methods  as  well.  Increasing 
the  K ratio  decreases  test  time  for  a given  level  of  risks  and  allowable 
number  of  failures.  However,  the  new  operating  characteristic  (O.C.) 
curve  adversely  affects  the  consumer  if  the  change  in  the  K ratio  is 
accomplished  by  decreasing  0i.  If  9o  is  modified  the  producer  is 
penalized.  In  this  report,  the  different  K ratios  were  obtained  by 
changing  6i,  analogous  to  the  procedure  used  in  MIL  STD  781-B. 

Although  all  test  methods  exhibit  this  sensitivity,  Bayes  plans 
utilizing  criteria  set  A-B  were  chosen  to  illustrate  this  effect.  Two 
additional  X values, K=l. 5 and  K=“3,  and  two  values  of  E{6),  E(6)=50  and 
E(6)“100,  were  utilized  in  this  analysis.  The  pertinent  graphs  are 
shown  in  Figures  29-34. 

It  is  to  be  observed  that  there  is  no  change  in  the  producer's  risk 
curves  as  a function  of  the  different  K vlues.  That  is,  the  family  of 
producer *8  risk  curves  for  E(9)«50  and  E(9,  ’’00  are  identical  for  K=1.5, 

2 and  3.  This  is  explained  by  recalling  that  the  change  in  the  discrim- 
ination ratio  results  from  a change  in  9i.  Since  the  producer's  risk 
does  not  involve  ^i,  this  function  is  not  affected.  However,  the  con- 
sumer's risk  function,  which  does  contain  6i  is  noticeably  different  as 
the  K ratio  is  varied.  As  in  the  Classical  Case,  the  change  is  in  a 
direction  which  requires  less  test  time  when  the  K value  is  increased, 
if  the  other  parameters  a’"e  held  constant.  The  graphs  for  E(0)-5O, 

Figures  29-31,  indicate,  for  example,  that  a test  plan  with  risks 
A-B"5Z  and  r*=0  requires  a T equal  to  one  9c  unic  when  the  K ratio  has  a 
value  of  3;  when  this  ratio  is  decreased  to  i.5,  test  time  increases  to 
5.5  00  units  while  the  ocher  parameters  are  kept  at  approximately  the 
same  values  (i.e.,  A-B=4.5Z  and  r**0) . 

The  consumer's  risk  curves  for  E(0)*i00,  Figures  31-34,  in  addition 
to  the  previously  noted  effects,  exhibit  different  initial  values  when 
Che  K ratio  is  varied  (i.e.,  have  different  values  of  consumer's  risk 
for  test  time  approaching  zero).  This  type  of  behavior  has  been  observed 
previously  but  under  different  conditions.  Earlier,  when  discussing  this 
result  in  connection  with  a Bayes  test,  0o  and  0i  remained  fixed  as  the 
prior  parameters  and,  therefore,  E(0)  Cook  on  different  values. 
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In  the  present  situation,  E(9)  end  6o  are  constant,  while  0 ; 
and  consequently,  K,  Is  varied.  Clearly  this  is  a different  condition; 
yet  one  which  is  amenable  to  a similar  explanation  providing  the  original 
context  is  widened.  The  following  rules  apply  to  the  more  general  case: 

a.  For  constant  E(6) 

An  increase  in  9o,  9i  or  both,  creates  a more  pessimistic 
prior  (i.e.,  it  decreases  the  chaice  of  successfully  passing  the  test); 
a decrease  is  9q,  0i  or  both,  produces  a more  optimistic  prior. 

b.  For  constant  0o,  9i 

An  increase  in  E{6)  leads  to  a more  optimistic  prior;  a de- 
crease in  E(0)  makes  the  prior  more  pes.simlstic. 

The  new  framework  helps  to  emphasize  that  the  prior  estimate  as 
well  as  the  test  specification  determine  the  classification  of  the  prior 
distribution.  The  graphs  of  Figures  13  and  34  tend  to  underscore  this 
fact.  As  can  be  observed,  the  initial  values  of  the  consumer’s  risk 
functions  are  similar  even  though  the  E(9)  and  9i  values  differ  con- 
siderably. The  explanation  is  that,  in  accordance  with  the  stated 
rules,  both  priors  are  to  be  classified  as  optimistic  priors.  This 
is  the  reason  for  the  similarity  in  the  initial  values  of  the  respective 
risk  functions  observable  on  the  graphc. 
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SECTION  V 
I.  TEST  PLAN  SELECTION  CRITERIA 

There  rre  two  fundamental  aspects  to  reliability  demonstration, 
economic  and  statistical.  Because  of  severe  resource  constraints, 
current  practice  is  to  choose  a test  based  primarily  on  economic 
considerations.  However,  statistical  issues  cannot  be  ignored  as 
they  affect  the  essential  purpose  and  usefulness  of  thia  type  of 
test. 


The  basic  aim  of  demonstration  testing  is  to  distinguish  between 
two  possible  values  in  a quality  characteristic  of  a product,  such  as 
MTBF.  For  the  Classical  method,  the  Instrument  that  measures  how  well 
a test  performs  this  function  is  called  the  Operating  Characteristic 
(O.C.)  curve  of  the  test.  The  curi'e  provides  a quantitative  method 
for  Judging  proposed  plans  and  has  long  served  as  a criterion  for  test 
plan  selection.  Unfortunately,  the  Bayes  approach  does  not  make  use 
of  a similar  procedure.  The  function  that  is  often  calculated  is  the 
probability  of  acceptance,  F(Acc),  a quantify  that  has  already  been 
discussed.  P(Acc)  is  the  classical  O.C.  curve  averaged  with  respect  to 
the  entire  prior  distribution  which,  for  a given  T,  r*  combination, 
takes  on  a single  value.  The  use  of  this  numeric  as  a figure  of  merit 
has  the  disadvantage  in  that  it  Includes  all  the  uncertainties  inherent 
in  the  prior  distribution.  Consequently,  information  Imparted  in  this 
way  tends  to  be  qualitative  and  more  difficult  to  interpret  compared  to 
what  is  available  using  the  conventional  approach.  Moreover,  as  test 
specifications,  for  all  methods  described,  are  stated  in  terns  of  specific 
values  of  MTBF  and  not  in  terms  of  distributions  or  averages,  it  seems 
appropriate  to  continue  use  of  O.C.  curves  as  a tool  for  the  evaluation 
and  comparison  of  any  test  plan,  no  matter  how  derived. 


One  approach  to  determine  t...  U.C.  curve  of  a test  plan  is  to 
substitute  the  T,  r*  values  of  that  plan  into  the  equation  for  P(Acc/6) 
(i.e.,  the  conditional  probability  of  acceptance  for  a given  value  of  6 
postulate  different  values  for  9 and  compute  P(Acc/8)  for  these  values. 
Since  a constant  hazard  rate  model  is  assumed  by  all  test  methods,  the 
appropriate  expression  for  P(Acc/S)  is  the  Pnissor.  function  summed  over 
the  allowable  number  of  failures.  Stated  in  equational  form. 


P(Acc/9)  * 


f=o 


where  T denotes  test  length;  r equals  the  number  of  failures  occurring 
in  T;  r*  is  the  maximum  number  of  failures  allowed  for  acceptance  and 
T/0  is  the  average  rate  of  failure  occurrences.  Thus,  once  the  pair 
(T,  r*)  of  a plan  is  known,  its  O.C.  curve  can  be  calculated.  .Also, 
since  a T,  r*  combination  determines  a unique  test  plan,  its  O.C.  curve 
is  also  unique.  This  approach,  therefore,  provides  a logical  and  objective 
basis  for  judging  how  well  expected  test  goals  are  being  achieved.  In 
sunmsary,  the  suggested  procedure  offers  the  following  advantages: 
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a.  It  is  consistent  with  the  form  in  which  the  test 
hypothesis  is  posed:  i.e.,  in  terms  of  single  MTBF  values,  not  in 
terms  of  distributions  or  averages.  The  Classical  O.C.  curve  is 
well  suited  to  provide  a concise  answer  to  a question  that  normally 
arises  in  this  context,  which  is:  how  does  the  proposed  test  perform  if 
the  product  has  an  MTBF  equal  to  the  value(s)  postulated? 

b.  It  is  not  directly  dependent  on  the  prior  distributional 
assumptions.  The  prior,  however,  partially  determines  the  T,r*  values 
which  are  used  to  compute  the  O.C.  curve. 


c.  It  provides  a common  basis  for  comparing  different  plans 
no  matter  how  derived. 

d.  It  is  simple  to  construct  a.»d  easy  to  evaluate. 

e.  It  permits  examination  of  a plan  in  terms  of  an  attribute 
whose  meaning  is  unambiguous  and  well  understood. 

Given  that  T,  r*  values  have  been  chosen,  the  actual  calculation 
of  the  O.C.  curve  can  be  accomplished  in  several  ways.  Since  the 
solution  involves  use  of  Poisson  probabilities,  which  are  widely 
tabulated  or  available  in  chart  format  (Thorndyke  chart),  a feasible 
method  is  to  reference  existing  aids  and  provide  instructions  on  their 
use.  This  approach,  however.  Introduces  some  inconvenience  in  per- 
forming the  necessary  computations  and  was,  therefore,  not  implemented. 
Instead,  it  was  decided  to  develop  a simple  graphic  scheme  capable  of 
addressing  this  problem  more  directly;  it  consists  of  plotting  contours 
of  constant  normalized  MTBF  values,  0/6o>  in  the  probability  of  accept- 
ance P(Acc)  and  normalized  test  time,  T/Oo,  plane.  Each  graph  contains 
a family  of  20  o/0(,  curves  which  vary  between  the  limits  of  .1  and  2, 
in  increments  o!  .1.  Also,  a separate  graph  is  constructed  for  each 
value  assigned  to  r*.  Ten  values  of  r*  were  selected,  starting  with  zero 
and  increasing  in  unit  steps  to  r*=9.  With  the  aid  of  these  graphs,  the 

0. C.  curve  of  any  plan  can  be  determined  by  means  of  a simple  two  step 
procedure: 

1.  Given  the  T,  r*  values  of  a plan,  select  the  graph  with  the  same 
r*  number  as  that  specified  by  the  plan;  (for  example,  if  .he  test  plan 
specifies  r*=6,  choose  Figure  Al). 

NOTE:  Two  different  symbols,  R*  in  the  graphs  and  r*  in  the  report,  have 
been  used  to  deonte  the  same  quantity.  .1.  R*=r* 

2.  Having  identified  the  proper  graph,  draw  a vertical  line, 

from  the  point  on  the  abscissa  equal  to  the  T value  of  the  test,  to 
intersect  all  20  curves  of  the  graph.  Each  point  of  intersection 
identifies  a value  for  the  acceptance  probability  which  can  be  read  on 
the  ordinate  scale.  The  9/Go  value  associated  with  a particular  point 
of  intersection  can  be  ascertained  as  follows:  The  first  O/Sq  curve, 

1. e.  the  one  closest  to  the  horizontal  axis,  has  a value  of  0/6o»-l; 
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next  curve  hae  a value  of  .2,  etc.  Thus,  an  O.C.  curve  can  be 
conatructcd  baaed  entirely  on  the  data  dt^rived  froa  the  20  points  of 
intersection.  The  following  exanple  will  illustrate  this  procedure 
it.  sore  detail. 

NOTE;  The  graphs  are  scaled  in  terms  of  nonaallzed  test  units.  Therefore, 
if  the  plan  measures  time  in  T/0o  units,  this  value  should  be  entered 
directly,  otherwise  the  specified  test  time  nnist  be  divided  by  0o’ 

Example  (8).  Example  2 of  this  report,  discusses  a Classical 
test  plan  with  the  following  characteristics:  a = 18Z;  8 = 12Z; 

T/00  * 6.3;  K * 2,  r*  = 8.  Determine  the  O.C.  curve  for  this  plan. 

Since  r*  = 8,  the  proper  graph  is  Figure  43.  Locate  the  T/6o  =6.3 
point  on  the  abscissa  of  that  graph.  Use  a ruler  to  draw  a vertical  line 
from  this  point  to  intersect  all  curves  In  that  graph.  A horizontal  line 
drawn  from  each  point  of  intersection  to  the  Y axis,  provides  the  corre- 
sponding values  of  the  probability  of  acceptance. 

The  data  is  most  suitably  arranged  in  terns  of  the  tabulation  shown 
below.  0/00  P(Acc)  percent 


.1 

0 

.2 

0 

.3 

0 

.4 

2.5 

.5 

12 

.6 

28 

.7 

48.5 

.8 

61 

.9 

73 

1.0 

81.5 

1.1 

86 

1.2 

91.5 

1.3 

]4 

1.4 

96 

1.5 

97 

1.6 

98 

1.7 

98.5 

1.8 

99 

1.9 

99.2 

2.0 

99.5 

The  O.C.  curve  of  this  test  can  be  constructed  entirely  from  the 
above  data.  However,  salient  test  properties  can  be  inferred  without 
actually  doing  this.  First  note  that  for  9/6o  = 1.0,  the  probability 
of  acceptance  equals  81.5%.  Since  the  test  was  constructed  to  have  an  a 
of  approximately  18%,  the  O.C.  data  shows  that  this  requirement  is 
approximately  satisfied  (i.e.,  since  P(Acc/0»0o)  = 1-a).  Similarly, 
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when  w/Oo  = -5,  the  corresponding  P(Acc)  value  is  12%.  This  correctly 
reflects  another  requireoent  of  this  plan,  which  is  that  S=12%=P(Acc/0=»9 j ) , 
when  the  discrimination  ratio  is  2.  Going  beyond  these  two  points,  the 
protection  afforded  by  this  test  can  also  be  ascertained  without 
difficulty.  The  region  of  main  concern  to  a consumer  encompasses 
acceptance  probabilities  for  small  values  of  9.  If  these  probabilities 
are  small,  this  indicates  that  he  is  getting  good  protection  from  the 
plan.  For  example,  a device  with  an  MTBF  of  40%  of  the  required  value 
will  only  have  a 2.5%  chance  of  being  accepted  by  the  test.  Conversely, 
the  portion  of  the  O.C.  curve  of  primary  interest  to  a producer  concern 
the  acceptance  probabilities  for  large  values  of  9.  The  data  Indicates 
that  a device  having  an  MTBF  that  is  120%  of  the  required  value  has  a 
91.5%  chance  of  being  accepted.  Tnis  O.C.  curve,  therefore,  reflects 
the  condition  that,  in  general,  good  devices  will  be  passed  and  poor 
devices  rejected  by  the  test,  consonant  with  the  basic  purpose  of  this 
type  of  test. 

Having  Illustrated  this  procedure  on  a Classical  Plan,  it  is 
informative  to  apply  the  technique  to  other  test  methods  as  well.  Since 
the  procedure  is  the  same  no  matter  what  plan  is  used,  the  intert-cdi."* ;e 
steps  are  omitted  and  only  final  results  are  shown.  Extending  the 
analysis  of  a previous  example  (example  5,  page  III-7),  the  O.C.  data  for 
each  plan  cited  in  example  5 was  calculated  and  is  presented  in  the 
following  table. 

O.C.  Tabulations  for  Test  Plans  of  Example  5 

KOTE:  The  common  features  of  these  test  plans  are:  Km2,  6-150; 

E(5)  = 7S;  0,5=100.  Characteristics  that  vary  from  plan  to  plan 


are  indicated  in  the 

data  column 

of  that  plan 

. 

A- 

B 

Ai-Bi 

A- 

»1 

A»B«8.6\ 

A»B«5.5% 

i.l«Bi»10.Sb 

Ai«B,«£.S\ 

A-B;-7.8\ 

A-B}«6t  ; 

r*«l 

t*-2 

r**3 

r*»4 

r*«l 

r*-2 

e/6o 

T/9o*-1.75 

T/90-2.1 

T/0O-2.4S 

T/eo-3.15 

CD 

O 

M 

T/eo-2.25 

.1 

0 

U 

0 

0 

0 

0 

.2 

0 

0 

0 

0 

0 

0 

.3 

2 

3 

4 

2 

4 

2 

.4 

7 

10 

14 

10.5 

11 

7.8 

.5 

13.5 

21 

28.0 

22 

20 

17 

.6 

21 

32 

42 

40 

29 

27.S 

.7 

29 

42.5 

53.5 

S3 

37 

37.5 

.8 

36 

51 

63.5 

64 

44 

47 

.9 

42 

59 

71 

72.5 

50 

55 

1.0 

47.5 

65 

77 

79 

56 

61 

1.1 

52.5 

70 

81.5 

84 

60 

66.5 

1.2 

57.5 

74.5 

85 

87.5 

65 

71 

1..3 

78 

88 

90 

68 

75 

1.4 

64 

81 

90 

92 

71 

78 

1.5 

67.5 

83.5 

92 

94 

73.5 

81 

1.6 

70 

85 

93 

95 

76 

83 

1.7 

72.5 

87 

94 

96 

78 

85 

1.8 

75 

88.5 

95 

96.5 

SO 

87 

1.9 

76.5 

90 

96 

97 

81.5 

88.5 

2.0 

78 

91 

96.5 

97.5 

83 

90 
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3y  way  of  comparison,  the  data  indicates  that  the  protection 
provided  by  these  plans  is  inferior  to  that  of  the  Classical  plan  of 
the  previous  example.  This  result  is  not  completely  unexpected  since 
the  test  time  for  the  latter  plan  is  considerably  greater  than  the 
times  required  by  the  Bayes  test-  Comparisons  within  Bayes  plans  identify 
plan  A-B  as  being  best  timewise,  but  lacking  in  protection  to  either 
consumer  or  producer.  Plan  Aj-Bi  does  better  for  the  producer  and  a 
little  worse  for  the  consumer.  The  poor  producer  protection  exhibited 
by  these  plans,  especially  plan  A-B,  is  somewhat  surprising  especially 
in  view  of  the  tendency  for  contractors  to  favor  the  Bayesian  approach. 
However,  this  data  cannot  be  considered  representative  since  it  is 
based  on  arbitrarily  selected  prior  parameter  values-  To  address  the 
more  general  condition,  the  O.C.  characteristics  of  four  Bayes  plans 
of  criteria  set  .A-B,  each  having  a different  prior,  were  calculated 
and  this  data  appears  in  the  following  table-  All  pertinent  test 
conditions  are  as  shown  in  the  table.  To  facilitate  comparisons,  an 
attempt  was  made  to  keep  the  risks  approximately  the  same. 


O.C.  Characteristics  for  Bayes  plans  using  criteria  set  A-B 

NOTE:  Common  features  of  the  plans  are  K-2;  Oo=100.  Other  conditions 

are  as  indicated  in  the  table. 

A«B-6»  A»B«8.6\  1 A-B-8.6\  A-B-5%  A-B-1.4% 

E(6)-S0  E(6)-75  E(0)«1OO  E(6)-150  E(B)-200 


. / 

.8 

.9 

1.0 

1 

1.1 

i 

1.2  I 

1.3  1 

* 

1.4  1 

l.S 

1.6 

1.7 

1 

1.8 

1.9 

1 

f 

2.0 
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This  tabulation  illustrates  a previously  nentioned  problem  with 
some  Bayes  tests  . It  was  pointed  out  on  page  II-5,  that,  if 
the  prior  is  overly  optimistic  about  the  capabl  .ities  of  a device,  a 
Bayes  procedure  can  be  very  poor  in  detecting  inadequate  devices.  The 
0-C-  data  In  the  last  column  (i.e.,  where  E(0)“200,  twice  the  required 
value)  confirms  this  statement.  The  data  shows,  for  example,  that  a 
device  having  only  iOX  of  the  required  MTBF  value  will  have  an  87.5 
chance  of  being  accepted  by  this  test.  The  adjacent  column,  with  a 
less  optimistic  prior,  does  better,  but  not  by  much.  For  C(9)=150, 
devices  with  an  MTBF  equal  to  50X  of  the  required  value,  will  be 
accepted  89%  of  the  time.  This  constitutes  very  poor  consumer  pro- 
tection, although  the  producer  does  benefit  in  that  alxx>st  nothing  gets 
rejected.  Resorting  to  plans  with  less  optimistic  priors  helps  the 
consumer,  but  this  gain  is  accomplished  at  the  expense  of  the  pro- 
ducer’s interest  as  his  protection  is  drastically  reduced.  This  is 


B 


exemplified  by  the  plan  having  an  £(6)  of  75;  that  plan  indicates  that 
a device  that  is  twice  as  good  as  the  required  MTBF  has  only  a 78% 
chance  of  being  accepted  by  the  test.  This,  of  course,  represents  an 
intolerable  situation  to  a contractor.  His  interest  is  better  served 
with  a more  optimistic  prior  but  unfortunately,  this  causes  consumer 
protection  to  deteriorate  rapidly,  as  has  been  shown.  In  general,  the 
use  of  an  extreme  prior  results  in  a test  which  favors  either  producer 
or  consumer,  depending  on  which  extreme  is  chosen. 

The  O.C.  data  f'or  several  additional  tests  previously  discussed  in 
this  report,  was  comr'led  to  permit  more  comprehensive  evaluation  of 
their  properties.  The  next  table  shown  is  for  plans  using  the  Aj-Bj 
criteria  set.  The  plans  also  differ  in  respect  to  values  assigned  to 
the  parameters  of  the  prior  distribution.  It  is  to  be  noted  that  these 
plans  provide  better  balanced  protection  than  the  A-B  plans  previously 
described. 
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The  O.C.  data  for  the  Important  Hybrid  test  category  has  been 
assembled  in  several  different  ways.  First,  the  data  for  the  Hybrid 
plans  of  example  8,  which  Involved  a comparison  of  plans  utilizing 
criteria  sets  A-8  and  Ax~8,  is  presented.  In  this  tabulation,  the  3rd 
column  contains  data  of  a comparable  Classical  plan.  Test  times,  risks 
and  otner  features  of  each  plan  are  as  Indicated  in  this  table.  Next, 
the  O.C.  characteristic  of  each  plan  is  depicted  as  a function  of  the 
prior  characteristics  and  is  presented  in  the  subsequent  two  tables. 

The  consumer  protection  provided  by  these  plans  is  generally  good  (of 
course,  at  the  g point,  the  risk  has  the  specified  value).  The  pro- 
ducer '§  interest  is  also  well  served  in  some  of  these  plans  especially 
when  the  prior  is  optimistic.  The  use  of  an  optiraistir  prior  in  this 
I test  method  does  not  lead  to  a breakdown  in  consumer  protection  as  it 

I does  in  some  Bayes  tests,  but  tends  to  be  of  mutual  benefit. 

E 

I 


O.C.  Tabulations  for  test  plans  of  Example  8 
Note:  the  common  features  of  these  plans  are:  K»2;  6“1S0;  E(0)»7S;  6q»100 


0/00 

Aj-6 

ApB=10,4% 

r*=6 

T/9o«5.25 

A-S 

A«B“9.5% 

r*-l 

T/0o-2 

oi-D 

a*B»15.3% 

r*-8 

T/00-6 

- 

.1 

0 

0 

0 

.2 

0 

0 

0 

.3 

0 

1 

0 

.4 

2.5 

4 

3.5 

.5 

10 

9 

15.5 

.6 

23 

15.5 

33.5 

.7 

37.5 

22 

51.5 

.8 

51.5 

29 

66 

9 

63 

35 

77 

1.0 

72.5 

40.5 

84.5 

1.1 

79.5 

46 

90 

' 1.2 

85 

50.5 

93 

i 1.3 

88 

54.5 

95 

t 1.4 

91.5 

58.5 

97 

i l.S 

93.5 

61.5 

97.5 

E 1.6 

95 

64.5 

98 

^ 1.7 

96 

67.5 

98.5 

1 1.8 

97 

70 

99 

f 1.9 

97. S 

72 

99.5 

1 2.0 

98 

74 

99.6 

Is 

! •; 
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O.C.  characteristics  of  Hybrid  plans  using  criteria  set  A-6 


Note:  common  characteristics  are:  K-2;  9^-100 


8/6o 

E(e)-50 

A“8*5.4l 

1=100 

T/eo«1.45 

r**0 

E(0)-75 

A-e«5.6% 

S-150 

4>»3 

7/00-4.5 

r*-4 

E(e)-100 

A-6»6.9\ 

S-200 

If*  3 

T/6o-5.65 

r*-6 

E(0)-150 
A-8-13.41> 
^■300 
<Jv3 
7/  0o®5 
r*-6 

E(0)-2OO 

A-6-21.4% 

S-400 

^•3 

7/00-4.5 

r*-6 

.1 

0 

0 

0 

0 

0 

.2 

0 

0 

0 

0 

0 

.3 

0 

0 

0 

0 

0 

.4 

2.5 

1 

1 

3 

7 

.5 

5.0 

S 

6.5 

13 

20.5 

T3 

575 

g 

11 

27.5 

37.' 

.7 

12.5 

23 

30 

43 

53.5 

.8 

16 

34 

44 

56.5 

66.5 

.9 

20 

44 

56 

68 

76 

— 

l.O 

23 

53.5 

66 

76 

83 

1.1 

26.5 

61 

74 

82.5 

88 

1.2 

30 

68 

80 

87 

91.5 

1.3 

32.5 

73.5 

85 

90 

93 

1.4 

35 

78 

88.5 

93 

95 

1.5 

37.5 

82.0 

91 

95 

97 

1.6 

40 

84.5 

93 

96 

97.5 

1.7 

42.5 

87 

95 

97 

98 

i.S 

45 

89 

96 

97.5 

98.5 

1.9 

47 

91 

96.5 

98 

99 

2.0 

48 

! 92.5 

97 

98.5 

99.5 
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FIGURE  3 - Bayes  Method;  Criteria  Set 


riGUW-  (■)  - Hybrid  Method  Criteria  Set  A-fi 


P'ICURE  7 - Hybrid  Method;  Criteria  Set  A^-}) 
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FIGURE  11  - Bayes  Method;  Criteria  Set  A-B 
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FIGURE  13  - Bayes  Method;  Criteria 


FIGURf:  14  - Bavos  Method 


FIGURE  IS  - Bayes  Method;  Criteria  Set  A 


Method;  Criteria  Set 


FIGURE  17  - Bayes  Method;  Criteria  Set  A 


FIGURH  18  - Bayes  Method;  Criteria  Set 


FIGURF.  21  - Hybrid  Methodj  Criteria  Set  A-p 
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Hybrid  Method;  Criteria  Set 


FIGURE  2S  - Hybrid  Method;  Criteria  Set  A 


d Method;  Criteria  Set  A 


FIGURE  27  - Hybrid  Method;  Criteria  Set  Aj^-p 


FIGURH  28  - Hybrid  Method;  Criteria  Sot 


FIGURE  29  - Bayes  Method;  Criteria  Set  A-B 


33  - Bayes  Method;  Critei'ia  Set  A-B 


ayes  Method;  Criteria  Sot  A-B 
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A r I’  I;  N [)  I \ A 
BASIC  I-ORMIJLAS 

When  the  time  to  failure  is  exponent  ia  ily  distributeii.  the  number 
vif  f.iilure-^  in  a jieriod  of  length  T has  the  I'oisson  distribution  with 
p.iraneter  I'/d,  so  that 

wliere  IMA  i is  the  conditional  probability  of  test  acceptance  (i.e., 
acceptance  of  the  hypothesis  that  0 = do)  jtiven  that  the  MTBI-  equals  9, 
and  I * IS  t'.e  naximum  number  of  fai  lures  permitted  in  tl.e  iaterai  T,  for 
an  I-, cept  decision. 


Cl.issical  test  plan  design  is  generally  based  on  the  following 
speci f icat ions: 


where  iijS  design. ite  the  classical  p.roJucer's,  consumer's  risk , 
respect ivcly . 
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hlun  :ht‘  prior  d i st  r ibul  n*n . f{Pi,  i'^  the  inxc-rtcd  g.imm.i  densjtv. 
It  I .in  i>i  e^prr'">ed 
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where  6,  c tlenotc  the  scale  aiiJ  shape  parameters  of  this  distribution, 
respectively,  and  are  assumed  to  be  known. 


subst  1 tu;  1 ni;  tiu-  expressions  ftir 
into  the  previous  eipi.'it  ion  yields 


comb i II  111 j;  teims  <^ives 


the  'u  vt  sv'.eral  stejis  invol.e  5 ransU»rm,it  ion  of  the  integral  to  an 
1 nci»r;p  i (.  I e garsa.’  function  b>  a change  of  v.iriables.  as  follows: 

0 =r  >C-"I 

siihsi  1 1 lit  I iig  these  varialiles  m the  integral  yieltls 
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subst itut ing 


The  aoovc  integral,  now  in  the  form  of  the  incomplete  gairana  functiop, 
can  he  solved  in  several  different  ways.  Since  tables  of  the  i nconnl etc 
ganna  fniution  .ire  available,  they  nav  be  used  to  obtain  a sohit  i •n;  f'eninr 
1 pg  -l"Ji  cites  se\er.il  other  procedures  for  evaliiat  iiig  this  ftin.  t i‘>n . 
bhen  ; i-*  ui  integer  > 0 the  solution  nay  !>e  derived  as  {f.Iiows. 
using  operator  notation: 


cj/y.  a (T*i)4X 
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pt r iorniiiig  the  indicated  operations  givf*s 


D 


ThF 


•f  Ml 


‘ w ^ 


. x0+r->  ^|-_j 
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P(e--e<,,Rej)=e'Vi-p/>  7^ 

^ ^ ^ OO 
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C'f*r~i)!  p*'(hp)^ 
(P-I).'f/  ' 


P(Acc)  = j - p(fej) 


A- 5 


J pXi-f,)' 


,1,  iiiiiiiiijibiu  %iu«iuLUliJiiLiliu:iliili.iliiildiidliiiiii^  lll,lliJyijHik’4iU  l^^iiliiiklJMliJiLliliiilg^ 


NAIX:-75265-60 


By  definition,  the  Bayes  consiiiaer's  risk  B is  expressed  as: 

g = P(Q-Q|j  Acc) 

P CAcc) 
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it  has  prevjoiisly  been  shown  (derivation  of  equation  5)  that 
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ind  for  ; = integer,  that: 
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stiiig  that 
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if  is  .'111  iiiteger,  this  expression  further  reduces  to 

OW"l  \ -'J 

where 
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Therefore 
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C^+r-0/  pY<-P)^ 
4 («>-)/ r/  ■ ^ 
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the  equation  for 


P(©2o«-) 


is  determined  as  follows: 
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this  integral  is  similar  to  the  one  appearing  in  the  derivation  of 
equation  A-4.  By  anoiogy; 
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equations  A-1  through  A-8  are  the  expressions  used,  in  various  combinations, 
to  compute  the  risk  functions  corresponding  to  the  values  assigned  to  T/0o 
and  r*.  These  data  were  then  incorporated  into  a plotting  routine  which 
generated  all  the  graphs  shown  in  the  body  of  the  report.  To  illustrate, 
graphs  of  the  risk  criteria  set  A - B were  formulated  using  the  following 
relationships: 

A - Eq^3tion  (A  - 3) 

Equation  (A  - 4) 

g _ Equation  (A  - 6) 

Equation  (A  - 5) 

for  criteria  set  the  following  equations  apply 

. _ .Equation  (A  - 3) 

^1  ” Equation  (A  - 7) 


R - Equation  (A  - 6) 

“l  ~ Equation  (A  - 8) 


for  criteria  set  A-B, 


Equation  (A  - 5) 
Equation  (A  - 4) 


B = Equation  (A  - 2) 


